Skip to content

Commit 800a97c

Browse files
a-zakira-zakir
authored
Merge pull request #15 from AntaresSimulatorTeam/feature/restore-repo
restore the repo
2 parents 9420b45 + 3c1ee31 commit 800a97c

7 files changed

Lines changed: 312 additions & 0 deletions

File tree

‎.github/workflows/generate-reference.yml‎

Lines changed: 144 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,144 @@
1+
name: Reference generation
2+
3+
on:
4+
# Manual launch
5+
workflow_dispatch:
6+
inputs:
7+
release_tag:
8+
description: "Release tag"
9+
required: true
10+
release_name:
11+
description: "Release name"
12+
required: true
13+
antares_tag:
14+
description: "Antares_Simulator solver tag"
15+
default: v9.0.0
16+
required: true
17+
antares_tests_tag:
18+
description: "Antares_Simulator_Tests_NR"
19+
default: v9.0.0
20+
required: true
21+
22+
jobs:
23+
release:
24+
runs-on: ubuntu-latest
25+
outputs:
26+
batches: ${{ steps.read_batches.outputs.batches }}
27+
28+
steps:
29+
- name: Checkout SimTest
30+
uses: actions/checkout@v4
31+
32+
- name: Create release
33+
env:
34+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
35+
tag: ${{ github.event.inputs.release_tag }}
36+
title: ${{ github.event.inputs.release_name }}
37+
antares_tag: ${{ github.event.inputs.antares_tag }}
38+
antares_tests_tag: ${{ github.event.inputs.antares_tests_tag }}
39+
run: |
40+
gh release create "$tag" \
41+
--repo="$GITHUB_REPOSITORY" \
42+
--title="$title" \
43+
--target="$target_branch" \
44+
--notes="Antares_Simulator: $antares_tag
45+
Antares_Simulator_Tests_NR: $antares_tests_tag"
46+
47+
- name: Download study-batches.txt
48+
run: |
49+
wget https://github.com/AntaresSimulatorTeam/Antares_Simulator_Tests_NR/releases/download/${{ github.event.inputs.antares_tests_tag }}/study-batches.txt
50+
51+
- name: Read study-batches.txt
52+
id: read_batches
53+
run: |
54+
BATCHES=$(printf "\"%s\"," $(cat study-batches.txt) | sed "s/^/[/;s/,$/]/")
55+
echo "batches=$BATCHES" >> $GITHUB_OUTPUT
56+
57+
- name: Upload study-batches.txt
58+
env:
59+
GITHUB_TOKEN: ${{ github.token }}
60+
tag: ${{ github.event.inputs.release_tag }}
61+
run: |
62+
gh release upload "$tag" study-batches.txt
63+
64+
generation:
65+
runs-on: ${{ matrix.os }}
66+
needs: release
67+
strategy:
68+
fail-fast: false
69+
matrix:
70+
batch: ${{ fromJson(needs.release.outputs.batches) }}
71+
os: [ubuntu-22.04, windows-2022]
72+
73+
steps:
74+
- name: Checkout SimTest
75+
uses: actions/checkout@v4
76+
77+
- name: Pre-requisites (Windows)
78+
if: ${{ startsWith(matrix.os, 'windows') }}
79+
run: choco install wget zip unzip --no-progress
80+
81+
- name: Download Antares_Simulator archive (Ubuntu)
82+
if: ${{ startsWith(matrix.os, 'ubuntu') }}
83+
run: |
84+
ANTARES_TAG=${{ github.event.inputs.antares_tag }}
85+
wget https://github.com/AntaresSimulatorTeam/Antares_Simulator/releases/download/${{ github.event.inputs.antares_tag }}/antares-${ANTARES_TAG/v/}-Ubuntu-22.04.tar.gz -O antares_simulator.tar.gz
86+
87+
- name: Download Antares_Simulator archive (Windows)
88+
if: ${{ startsWith(matrix.os, 'windows') }}
89+
run: |
90+
ANTARES_TAG=${{ github.event.inputs.antares_tag }}
91+
wget https://github.com/AntaresSimulatorTeam/Antares_Simulator/releases/download/${{ github.event.inputs.antares_tag }}/antares-solver_windows.zip -O antares_simulator.zip
92+
shell: bash
93+
94+
- name: Unpack Antares_Simulator, clean archive (Ubuntu)
95+
if: ${{ startsWith(matrix.os, 'ubuntu') }}
96+
run: |
97+
tar xf antares_simulator.tar.gz
98+
rm antares_simulator.tar.gz
99+
100+
- name: Unpack Antares_Simulator, clean archive (Windows)
101+
if: ${{ startsWith(matrix.os, 'windows') }}
102+
run: |
103+
unzip -q antares_simulator.zip
104+
rm antares_simulator.zip
105+
106+
- name: Download Antares_Simulator_Tests
107+
run: wget https://github.com/AntaresSimulatorTeam/Antares_Simulator_Tests_NR/releases/download/${{ github.event.inputs.antares_tests_tag }}/${{ matrix.batch }}.zip -O input.zip
108+
109+
- name: Unpack test studies, clean archive
110+
run: |
111+
unzip -q input.zip
112+
rm input.zip
113+
114+
- name: Generate results (Ubuntu)
115+
if: ${{ startsWith(matrix.os, 'ubuntu') }}
116+
run: |
117+
ANTARES_TAG=${{ github.event.inputs.antares_tag }}
118+
python3 scripts/generate_reference.py ${{ matrix.batch }} antares-${ANTARES_TAG/v/}-Ubuntu-22.04/bin/antares-solver
119+
120+
- name: Generate results (Windows)
121+
if: ${{ startsWith(matrix.os, 'windows') }}
122+
run: |
123+
ANTARES_TAG=${{ github.event.inputs.antares_tag }}
124+
python3 scripts/generate_reference.py ${{ matrix.batch }} antares-solver.exe
125+
shell: bash
126+
127+
- name: Write tags to file
128+
shell: bash
129+
run: |
130+
echo Antares_Simulator ${{ github.event.inputs.antares_tag }} >> ${{ matrix.batch }}/version.txt
131+
echo Antares_Simulator_Tests_NR ${{ github.event.inputs.antares_tests_tag }} >> ${{ matrix.batch }}/version.txt
132+
echo SimTest ${{ github.event.inputs.release_tag }} >> ${{ matrix.batch }}/version.txt
133+
134+
- name: zip results
135+
run: zip -q -r ${{ matrix.batch }}-${{ matrix.os }}.zip ${{ matrix.batch }}
136+
137+
- name: Upload .zip
138+
env:
139+
GITHUB_TOKEN: ${{ github.token }}
140+
tag: ${{ github.event.inputs.release_tag }}
141+
shell: bash
142+
run: |
143+
gh release upload "$tag" ${{ matrix.batch }}-${{ matrix.os }}.zip
144+

‎README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
# SimTest
2+
Central repo for Antares_Simulator tests

‎requirements.txt‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
pytest
2+
pytest-xdist
3+
pandas

‎scripts/antares_test_utils.py‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
from pathlib import Path
2+
import glob
3+
import shutil
4+
5+
from study import Study
6+
7+
def list_directories(directory):
8+
dir_path = Path(directory)
9+
assert(dir_path.is_dir())
10+
dir_list = []
11+
for x in dir_path.iterdir():
12+
if x.is_dir():
13+
dir_list.append(x)
14+
return dir_list
15+
16+
def find_studies_in_batch_dir(batch_name):
17+
batch_directory = Path(batch_name).resolve()
18+
studies = []
19+
if (batch_directory.is_dir()):
20+
study = Study(batch_directory)
21+
if study.check_files_existence():
22+
studies.append(batch_directory)
23+
else:
24+
for x in batch_directory.iterdir():
25+
studies.extend(find_studies_in_batch_dir(x))
26+
27+
return studies
28+
29+
def get_headers(df) -> set :
30+
return set(df.columns)
31+
32+
def remove_possibly_remaining_outputs(study_path):
33+
output_path = study_path / 'output'
34+
shutil.rmtree(output_path, ignore_errors=True)
35+
36+
def move_output_to_reference(study_path):
37+
output_path = study_path / 'output'
38+
list_dir = list_directories(output_path) # list of 'output' sub-directories
39+
if len(list_dir) != 1 : # We should have only 1 results directory
40+
raise AssertionError("Too many results directories in output")
41+
result_dir = list_dir[0]
42+
reference_path = study_path / 'output' / 'reference'
43+
shutil.move(result_dir, reference_path)
44+
45+
46+
def enable_study_output(study_path, enable):
47+
st = Study(study_path)
48+
if not st.check_files_existence():
49+
raise RuntimeError("Missing file")
50+
51+
synthesis_value = "true" if enable else "false"
52+
st.set_variable(variable = "synthesis", value = synthesis_value, file_nick_name="general")
53+

‎scripts/generate_reference.py‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
import argparse
2+
3+
import antares_test_utils as antares_utils
4+
from run_command_building import *
5+
import sys
6+
7+
# Parsing args
8+
parser = argparse.ArgumentParser()
9+
parser.add_argument("batch_name", help="Batch directory (containing studies)",
10+
type=str)
11+
parser.add_argument("path_where_to_find_exe", help="Path to executable",
12+
type=str)
13+
# Storing args
14+
args = parser.parse_args()
15+
batch_name = args.batch_name
16+
path_where_to_find_exe = Path(args.path_where_to_find_exe)
17+
18+
if not path_where_to_find_exe.is_dir() and not path_where_to_find_exe.is_file():
19+
raise RuntimeError("Path where to find an executable does not exist")
20+
21+
22+
# Looking for studies in batch directory
23+
study_path_collection = antares_utils.find_studies_in_batch_dir(batch_name)
24+
25+
ret = []
26+
for study_path in study_path_collection:
27+
print("Study : %s ... " % study_path.name)
28+
29+
antares_utils.remove_possibly_remaining_outputs(study_path)
30+
antares_utils.enable_study_output(study_path, True)
31+
32+
command_to_run = make_command_to_run(path_where_to_find_exe, batch_name, study_path)
33+
result = run_command(command_to_run)
34+
ret.append(result)
35+
if result:
36+
print('Run study OK')
37+
else:
38+
print ('Error while running study, aborting')
39+
exit(1)
40+
41+
antares_utils.move_output_to_reference(study_path)
42+
43+
sys.exit(not all(ret))

‎scripts/run_command_building.py‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
from pathlib import Path
2+
import subprocess
3+
import sys
4+
5+
def make_command_to_run(path_where_to_find_exe, batch_name, study_path):
6+
command_to_run = []
7+
exe_path = Path()
8+
exe_identifier = "solver" # Default value
9+
10+
exe_path = path_where_to_find_exe.resolve()
11+
print(f"Found executabled : {exe_path}")
12+
13+
command_to_run = [exe_path, "-i", str(study_path)]
14+
if batch_name == "valid-milp":
15+
command_to_run.append('--solver=coin')
16+
17+
if batch_name == "valid-named-mps":
18+
command_to_run.append('--named-mps-problems')
19+
20+
return command_to_run
21+
22+
def run_command(command):
23+
process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=None)
24+
process.communicate()
25+
exit_code = process.wait()
26+
return (exit_code == 0)

‎scripts/study.py‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
from pathlib import Path
2+
3+
class Study:
4+
"""
5+
Class Study
6+
"""
7+
def __init__(self, dir_path):
8+
assert isinstance(dir_path, Path)
9+
self.study_dir = dir_path
10+
self.files_path = {}
11+
self.files_path["desktop"] = self.study_dir / "Desktop.ini"
12+
self.files_path["general"] = self.study_dir / "settings" / "generaldata.ini"
13+
self.files_path["study"] = self.study_dir / "study.antares"
14+
15+
def check_files_existence(self):
16+
return all([x.is_file() for x in self.files_path.values()])
17+
18+
def set_variable(self, variable, value, file_nick_name):
19+
"""
20+
Setting variable with a value in a file
21+
"""
22+
# File path
23+
file = self.files_path[file_nick_name]
24+
25+
# Content to print in file (tmp content)
26+
content_out = []
27+
28+
# Reading the file content (content in)
29+
with open(file) as f:
30+
content_in = f.readlines()
31+
32+
# Searching variable and setting its value in a tmp content
33+
for line in content_in:
34+
if line.strip().startswith(variable):
35+
content_out.append(variable + " = " + value + "\n")
36+
else:
37+
content_out.append(line)
38+
39+
# Erasing file content with the tmp content (content out)
40+
with open(file, "w") as f:
41+
f.writelines(content_out)

0 commit comments

Comments
 (0)