refactor tutorial structure and update for recent EDM4hep changes

fast-sim-and-analysis/FccFastSimGeneration.md → 2-gen-and-fastsim/2-1-event-generation/README.md

@@ -1,4 +1,4 @@
-# FCC: Getting started with event generation
+# Getting Started with Event Generation
 
 >
 > Original author: Gerardo Ganis
@@ -1193,7 +1193,7 @@ Now we are ready to go.
 
 #### Creating histograms with FCCAnalyses
 At this purpose we will use the recently introduced `build_graph` attribute. The example is availble at 
-[histmaker_ttmm.py](https://fccsw.web.cern.ch/tutorials/apr2023/tutorial1/histmaker_ttmm.py).
+[histmaker_ttmm.py](./histmaker_ttmm.py).
 
 ##### Dissection of `histmaker_ttmm.py`
 
@@ -1282,7 +1282,7 @@ The `build_graph` is part of the `fccanalyses run`:
 ```bash
 fccanalysis run histmaker_ttmm.py
 ```
-This should produce `ROTO` files with the histograms in `./outputs`:
+This should produce `ROOT` files with the histograms in `./outputs`:
 ```bash
 ls -lt outputs/
 ```
@@ -1302,9 +1302,9 @@ fccanalysis plots plots_ttmm.py
 ```
 with results available under `plots`.
 
-Example of a result are: [positive muon momentum](images/p_mup.png),
-[positive muon cosine theta](images/costheta_mup.png),
-[acollinearity of muons](images/acolmu.png).
+Example of a result are: [positive muon momentum](p_mup.png),
+[positive muon cosine theta](costheta_mup.png),
+[acollinearity of muons](acolmu.png).
 
 
 #### Possible conclusion of the exercise

2-gen-and-fastsim/2-1-event-generation/histmaker_ttmm.py

@@ -0,0 +1,92 @@
+
+# list of processes (mandatory)
+processList = {
+    'p8_tautau_ecm91':    {'fraction':1},
+    'wz_tautau_ecm91':    {'fraction':1},
+    'kk_tautau_ecm91':    {'fraction':1},
+}
+# Link to the dictonary that contains all the cross section informations etc...
+# Mandatory but anyone is good for local files
+procDict = "FCCee_procDict_winter2023_IDEA.json"
+
+# Define the input dir (optional)
+inputDir    = "./generation/gen/"
+
+#Optional: output directory, default is local running directory
+outputDir   = "./outputs/histmaker/ttmm/"
+
+# optional: ncpus, default is 4, -1 uses all cores available
+nCPUS       = -1
+
+# scale the histograms with the cross-section and integrated luminosity
+doScale = False
+# doScale = True
+# intLumi = 1000000 # 1 /ab
+
+# define some binning for various histograms
+bins_p_l = (100, 0, 50) # 0.5 GeV bins
+bins_cosTheta = (50, -1, 1)
+bins_acol = (50, -1, -.9)
+
+# build_graph function that contains the analysis logic, cuts and histograms (mandatory)
+def build_graph(df, dataset):
+    import ROOT
+    ROOT.gInterpreter.Declare("""
+
+       #ifndef funDone
+       #define funDone
+
+       float cosTheta(const edm4hep::Vector3d& in){
+          return (in.z/sqrt(pow(in.x,2)+pow(in.y,2)+pow(in.z,2)));
+       };
+
+       float scalarProductNorm(const edm4hep::Vector3d& in1, const edm4hep::Vector3d& in2 ){
+          return ((in1.x*in2.x + in1.y*in2.y + in1.z*in2.z)/sqrt(pow(in1.x,2)+pow(in1.y,2)+pow(in1.z,2))/sqrt(pow(in2.x,2)+pow(in2.y,2)+pow(in2.z,2) ));
+       };
+
+       float momP(const edm4hep::Vector3d& in ){
+          return (sqrt(pow(in.x,2)+pow(in.y,2)+pow(in.z,2)));
+       };
+
+       #endif
+    """)
+
+    results = []
+    df = df.Define("weight", "1.0")
+    weightsum = df.Sum("weight")
+
+    # Connect to taus
+    df = df.Define("tauplusvec", ROOT.MCParticle.sel_pdgID(-15, 0),["MCParticles"])
+    df = df.Define("tauminusvec", ROOT.MCParticle.sel_pdgID(15, 0),["MCParticles"])
+    df = df.Define("tauplus", "tauplusvec[0]")
+    df = df.Define("tauminus", "tauminusvec[0]")
+
+    # Connect to muons
+    df = df.Define("muplusvec", ROOT.MCParticle.sel_pdgID(-13, 0),["MCParticles"])
+    df = df.Define("muminusvec", ROOT.MCParticle.sel_pdgID(13, 0),["MCParticles"])
+    df = df.Define("muplus", "muplusvec[0]")
+    df = df.Define("muminus", "muminusvec[0]")
+
+    # CosThetas
+    df = df.Define("cthetaup", "cosTheta(tauplusvec[0].momentum)")
+    df = df.Define("cthetaum", "cosTheta(tauminusvec[0].momentum)")
+    df = df.Define("cthemup", "cosTheta(muplusvec[0].momentum)")
+    df = df.Define("cthemun", "cosTheta(muminusvec[0].momentum)")
+
+    # Acollinearities
+    df = df.Define("acoltau", "scalarProductNorm(tauminusvec[0].momentum, tauplusvec[0].momentum)")
+    df = df.Define("acolmu", "scalarProductNorm(muminusvec[0].momentum, muplusvec[0].momentum)")
+
+    # Muon momenta
+    df = df.Define("muplus_p", "momP(muplusvec[0].momentum)")
+    df = df.Define("muminus_p", "momP(muminusvec[0].momentum)")
+
+    # baseline histograms, before any selection cuts (store with _cut0)
+    results.append(df.Histo1D(("P_mup", "", *bins_p_l), "muplus_p"))
+    results.append(df.Histo1D(("P_mum", "", *bins_p_l), "muminus_p"))
+    results.append(df.Histo1D(("CosTheta_taup", "", *bins_cosTheta), "cthetaup"))
+    results.append(df.Histo1D(("CosTheta_mup", "", *bins_cosTheta), "cthemup"))
+    results.append(df.Histo1D(("AcolTau", "", *bins_acol), "acoltau"))
+    results.append(df.Histo1D(("AcolMu", "", *bins_acol), "acolmu"))
+
+    return results, weightsum

fast-sim-and-analysis/k4simdelphes/doc/starterkit/FccFastSimDelphes/Readme.md → 2-gen-and-fastsim/2-2-fastsim-delphes/README.md

@@ -1,4 +1,4 @@
-# FCC: Getting started with simulating events in Delphes
+# Getting Started with Fast Simulation in Delphes
 
 :::{admonition} Learning Objectives
 :class: objectives

fast-sim-and-analysis/eedE.md → 2-gen-and-fastsim/2-4-eedE/README.md

@@ -1,4 +1,4 @@
-Understanding generated process: eedE
+EDM4hep Event Data Explorer: eedE
 ===========================================
 
 The [EDM4hep event data explorer (eedE)](https://key4hep.github.io/eede/release/index.html) is a tool for visualizing the association between various objects in EDM4hep events. It is lightweight and self-explanatory. This section explains the usage of eedE. 
@@ -38,7 +38,7 @@ An example output can be found at [example.edm4hep.json](https://fccsw.web.cern.
 ## Using eedE
 
 Once the data has been converted into a json format via edm4hep2json, one can then head to the website of [eedE](https://key4hep.github.io/eede/release/index.html). After pressing the Start button, one is required to upload the EDM4hep json file via the Browse button. You can then select the type of association (`view`) to visualize.
-```{image} images/eedE/eede_upload.png
+```{image} eede_upload.png
 :align: center
 :width: 600px
 ```
@@ -47,7 +47,7 @@ Once the data has been converted into a json format via edm4hep2json, one can th
 
 Here we take the MC particle tree as an example.
 In the tree shown in the picture illustrates a collision at 91 GeV, where both the electron and positron emit a ISR photon before they merge into an on-shell Z boson, which decays into a pair of b quarks.
-```{image} images/eedE/eede_Zbb_example.png
+```{image} eede_Zbb_example.png
 :align: center
 :width: 400px
 ```

2-gen-and-fastsim/README.md

@@ -0,0 +1,24 @@
+# Generation and Fast Simulation
+
+If you want to get started with generation and a first quick inspection of fast-simulated
+events, you're at the right place. This chapter focuses on the generation of samples. A more in-depth look at FCCAnalyse is given in the next chapter.
+
+Fast simulation is currently supported through the Delphes approach. Support for the Papas approach, initially used for FCC-ee, is
+discontinued.
+
+If you have any problems or questions, you can
+[open an issue][fcc-tutorials-issues] on the
+[GitHub repository where these lessons are developed][fcc-tutorials-repo].
+
+
+[fcc-tutorials-issues]: https://github.com/HEP-FCC/fcc-tutorials/issues
+[fcc-tutorials-repo]: https://github.com/HEP-FCC/fcc-tutorials
+
+```{eval-rst}
+.. toctree::
+    :caption: Contents:
+
+    2-1-event-generation/README.md
+    2-2-fastsim-delphes/README.md
+    2-4-eedE/README.md
+```

3-analysis/3-1-higgs-analysis/README.md

@@ -0,0 +1,146 @@
+# Getting Started: Higgs Mass Analysis with FCCAnalyses
+
+>
+> Original author: Michele Selvaggi
+>
+
+Let's first clone and build `FCCAnalyses` with the following commands:
+```bash
+git clone --branch master https://github.com/HEP-FCC/FCCAnalyses.git
+cd FCCAnalyses
+source ./setup.sh
+fccanalysis build -j 8
+```
+and create a directory containing this tutorial:
+```bash
+cd ..
+mkdir tutorial && cd tutorial
+```
+
+Copy the necessary files, either from the web or from some location where the
+files were produced locally
+
+```bash
+mkdir localSamples && cd localSamples
+mkdir  p8_ee_WW_mumu_ecm240 && cd p8_ee_WW_mumu_ecm240
+wget https://fccsw.web.cern.ch/tutorials/ana-sim-evt/p8_ee_WW_mumu_ecm240/p8_ee_WW_mumu_ecm240_edm4hep.root
+cd ..
+mkdir p8_ee_ZZ_mumubb_ecm240 && cd p8_ee_ZZ_mumubb_ecm240
+wget https://fccsw.web.cern.ch/tutorials/ana-sim-evt/p8_ee_ZZ_mumubb_ecm240/p8_ee_ZZ_mumubb_ecm240_edm4hep.root
+cd ..
+mkdir p8_ee_ZH_Zmumu_ecm240  && cd p8_ee_ZH_Zmumu_ecm240
+wget https://fccsw.web.cern.ch/tutorials/ana-sim-evt/p8_ee_ZH_Zmumu_ecm240/p8_ee_ZH_Zmumu_ecm240_edm4hep.root
+cd ../..
+```
+
+This tutorial consists in two parts. Both parts will make use of ee->ZH (Z->mumu)events, with its relevant backgrounds ee->WW and ee->ZZ:
+
+- in **Part I** you will construct the recoil mass observable and apply a list of pre-selection cuts.
+- in **Part II** you will learn how to run an exclusive jet clustering algorithm, evaluate the various jet flavor probabilities and use them to select H->bb events.
+
+:::{admonition} Learning Objectives
+:class: objectives
+
+In this first example, you will learn how to: 
+
+-   read the **edm4hep** data format and construct physics observable (such as the recoil mass)
+-  define C++ helper functions in a separate that will be compiled at run time 
+-   apply an **event selection** and **fill histograms** in a single iteration using the **histmaker** option.
+-   produce **flat ntuples** with observables of interest with **FCCAnalyses**
+-   produce plots with the **plot** option. 
+:::
+
+## Part I - Recoil Mass: Analysis and Histograms in a Single Step
+
+Start by downloading this file called [functions.h](functions.h). This contains C++ helper functions that you will need to compute observables with RDataframe.
+
+Next, download the [histmaker_recoil.py](histmaker_recoil.py) script, which processes the previously produced ROOT samples, reconstructs muon and Z candidates, applies a sequence of event-selection cuts, and produces a set of histograms. 
+
+The script first defines the input/output paths, luminosity scaling, and histogram binning. The main analysis logic is implemented in the ```build_graph``` function: reconstructed muons are selected and isolated muons with ```p > 20 GeV``` are retained. The analysis then applies successive cuts, including opposite-sign muon selection, Z-boson candidate reconstruction, Z mass and momentum requirements, missing-energy selection, and finally a recoil-mass window to suppress background processes.
+
+Before running the script, update the ```inputDir``` variable so it points to the directory containing your local samples; the output histograms will be written to ```outputDir```. Finally, run the histmaker script using the fccanalysis ```run``` parameter to generate a ROOT file containing the resulting TH1 histograms:
+
+```bash
+fccanalysis run histmaker_recoil.py
+```
+
+Now download the the plotting script [plots_recoil.py](plots_recoil.py) and run it using the fccanalysis ```plots``` parameter:
+
+
+```bash
+fccanalysis plots plots_recoil.py
+```
+
+Please note that the event statistics is not great because we only produced on 10 000 events in the `Delphes Fast Simulation` step.
+
+
+## Part II - Jet Flavor: Producing a flat ntuple tree before creating histograms and plots
+
+:::{admonition} Learning Objectives
+:class: objectives
+
+In this second example, you will learn how to: 
+
+-  read the **edm4hep** data format, produce jet collections and evaluate the ParticleNet jet tagger score and apply an an event **preselection**
+-  produce **flat ntuples** with observables of interest with **FCCAnalyses**
+-  apply an **event selection** and **fill histograms** in a single iteration using the **histmaker** option.
+-   produce plots with the **plot** option. 
+:::
+
+
+This first analysis stage usually runs on large samples on batch, and the idea is to produce small ntuples with less variables. Also, jet inference of the jet tagger is pretty slow, and usually has to be performed only once. In this exercise we will run the **histmaker** in a separate, second stage. 
+
+Download this analysis script [treemaker_flavor.py](treemaker_flavor.py) and run
+
+```bash
+fccanalysis run treemaker_flavor.py
+```
+
+This will produce a root file for every process containing a flat tree `events` containing high-level variables. In addition to selecting muons as done in the previous example, this script create a new collection of `ReconstructedParticles` without the two high energy muons, and runs the jet clustering algorithm in exclusive N=2 mode of the new collection. In such a way, the two clustered jets form the H->jj jet candidates. The jet flavour algorithm is then evaluated on these two jets and the relevant scores are stored in the output tree.   
+
+Now we can download a dedicated [histmaker_flavor.py](histmaker_flavor.py) script, that runs this time on the just created flat ntuples files instead of the edm4hep format.
+
+Run it as before: 
+
+```bash
+fccanalysis run histmaker_flavor.py
+```
+
+This script takes the ntuples produced in the previous step, applies a similar selection as in the previous examples, but requires the jets to have a high probability to be B-like. 
+
+To compare the results between the flavor and recoil approaches, we again produce plots using a dedicated [plots_flavor.py](plots_flavor.py) script and run it:
+
+```bash
+fccanalysis plots plots_flavor.py
+```
+
+:::{admonition} Exercises
+:class: challenge
+
+## Simple
+
+1) Modify `histmaker_flavor.py` to require the two jets individually to be B-like, i.e requiring the B score is greater 0.5 for each jet. 
+
+2) Selecting gluon-like jets instead. You will need to modify both the  `treemaker_flavor.py` and `histmaker_flavor.py` for this.)
+
+3) Produce plots with larger statistics by re-running `DelphesPythia8_EDM4HEP` with more events. In particular produce a ZZ inclusive sample using to include all Z decays. Rerun all the examples above.
+
+## Advanced
+
+1) To evaluate the impact of detector performance, smear the neutral hadron resolution in the `ReconstructedParticlesNoMuons` collection and check the impact on the dijet invariant mass resolution. An example can be found [here] (https://github.com/HEP-FCC/FCCAnalyses/blob/master/examples/FCCee/smearing/smear_jets.py):
+
+2) Now smear the impact parameter resolution and evaluate the imapct on the efficiency of selecting two B-tagged jets or two C-tagged jets. 
+
+3) **This part can only be on lxplus and for people having the access rights to eos and the analysis dictonary**
+In order to produce plots with more statistics using centrally produced samples, we could use already processed large statistics samples.
+To do so we re-run the pre-selection over 10 percent of the total statistics [here](https://fcc-physics-events.web.cern.ch/FCCee/spring2021/Delphesevents_IDEA.php).
+Add to your a `analysis_stage1.py` file
+
+```python
+processList = {
+    'p8_ee_ZZ_ecm240':{'fraction':0.1},
+    'p8_ee_WW_ecm240':{'fraction':0.1},
+    'p8_ee_ZH_ecm240':{'fraction':0.1}
+}
+prodTag     = "FCCee/winter2023/IDEA/"
+:::