forked from simgrid/simgrid
-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathChangeLog
More file actions
4008 lines (3458 loc) · 182 KB
/
ChangeLog
File metadata and controls
4008 lines (3458 loc) · 182 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
----------------------------------------------------------------------------
SimGrid (3.23.3) NOT RELEASED YET (v3.24 expected September 23. 7:50 UTC)
S4U:
- Barrier::wait returns SG_BARRIER_SERIAL_THREAD for (only) one actor
for consistency with pthread_barrier_wait()
- Host::get_englobing_zone() returns the englobing netzone
- Actor::on_destruction is now called in the destructor
Actor::on_termination new signal called when the actor terminates
its code.
MSG:
- convert a new set of functions to the S4U C interface and move the old MSG
versions to legacy (MSG_process_self*, MSG_process_{un}ref, ...)
SMPI:
- Fortran bindings for DVFS have been removed.
- Add support for MPI_Irsend, MPI_Rsend, MPI_Rsend_init
- SMPI can now be selected by cmake's find_module(MPI) with
MPI_C_COMPILER, MPI_CXX_COMPILER, MPI_Fortran_COMPILER variables.
- Add support for MPI Errhandlers in Comm, File or Win. Default errhandler is now
MPI_ERRORS_ARE_FATAL, so codes which were sending warnings may start failing.
Model-Checker:
- Option model-checker/hash was removed. This is always activated now.
- New option smpi/buffering controls the MPI buffering in MC mode.
- MPI calls now MC_assert() that no MPI_ERR_* code is returned.
This is useful to check for MPI compliance.
XBT:
- xbt_mutex_t and xbt_cond_t are now marked as deprecated, a new C interface
on S4U is already available to replace them by sg_mutex_t and sg_cond_t.
Bugs:
- FG#28: add sg_actor_self (and other wrappers on this_actor methods)
- FG#29 and FG#33: provide a new C API to mutexes and condition variables
- FG#30: convert MSG_process_{un}ref to sg_actor_{un}ref
- FG#34: SG_BARRIER_SERIAL_THREAD?
- FG#35: model-checker does not like buster-produced binaries
----------------------------------------------------------------------------
SimGrid (3.23.2) July 8. 2019
Documentation:
- Nicer introduction page.
- Migrate the "Deploy your application" page to the new doc.
- Move Java as a subtree of MSG.
General:
- Rename simgrid::TimeoutError to simgrid::TimeoutException.
XBT:
- Drop xbt_dynar_sort_strings().
Bugs:
- Really fix FG#26: Turning off a link should raise NetworkFailureException
- FG#27: Wrong exception thrown to wait_any when link is turned off
- GH#328: Java: Canceling multiple tasks in a single vm/host
----------------------------------------------------------------------------
SimGrid (3.23) June 25. 2019
The Exotic Solstice Release.
General:
- SunOS and Haiku OS support. Because exotic platforms are fun.
- Stop setting random seed with srand48() at initialization.
- Use addr2line as a fallback for stacktraces when backtrace is not available.
- Build option -Denable_documentation is now OFF by default.
- Network model 'NS3' was renamed into 'ns-3'.
Python:
- Simgrid can now hopefully be installed with pip.
S4U:
- wait_any can now be used for asynchronous executions too.
XBT:
- New log appenders: stdout and stderr. Use stdout for xbt_help.
- Drop xbt_dict_dump.
SMPI:
- SMPI now reports support of MPI3.1. This does not mean SMPI supports all MPI 3 calls, but it was already the case with 2.2
- MPI/IO is now supported over the Storage API (no files are written or read, storage is simulated). Supported calls are all synchronous ones.
- MPI interface is now const correct for input parameters
- MPI_Ireduce, MPI_Iallreduce, MPI_Iscan, MPI_Iexscan, MPI_Ireduce_scatter, MPI_Ireduce_scatter_block support
- Fortran bindings for async collectives.
- MPI_Comm_get_name, MPI_Comm_set_name, MPI_Count support.
Model-checker:
- Remove option 'model-check/record': Paths are recorded in any cases now.
- Remove option 'model-check/sparse-checkpoint': Checkpoints are now
always sparse. That's an immense gain in memory, and can even be
faster because of cache effects. So there is no need to clutter the
code to allow the user to go for the unefficient mode.
Network models:
- Remove the lagrange-based models (Reno/Reno2/Vegas). The regular
models proved to be more accurate than these old experiments.
Fixed bugs (FG=FramaGit; GH=GitHub -- Please prefer framagit for new bugs)
- FG#1: Broken link in error messages
- FG#2: missing installation documentation
- FG#3: missing documentation in smpirun
- FG#6: Python bindings not available on PyPI
- FG#7: simple cmake call requires doxygen
- FG#8: make python bindings an optional dependency
- FG#10: Can not use MSG_process_set_data from SMPI any more
- FG#11: Auto-restart actors forget their on_exit behavior
- FG#12: -Denable_lto=OFF doesn't disable LTO
- FG#13: Installs unstripped file 'bin/graphicator'
- FG#14: Installs the empty directory 'doc/simgrid/html'
- FG#15: Setting -Denable_python=OFF doesn't disable the search for pybind11
- FG#17: Dead link in doc (pls_ns3)
- FG#20: 'tesh --help' should return 0
- FG#21: Documentation link on http://simgrid.org/ broken
- FG#22: Debian installation instruction are broken
- FG#26: Turning off a link should raise NetworkFailureException exceptions
- GH#133: Java: a process can run on a VM even if its host is off
- GH#320: Stacktrace: Avoid the backtrace variant of Boost.Stacktrace?
- GH#326: Valgrind-detected error for join() when energy plugin is activated
----------------------------------------------------------------------------
SimGrid (3.22) April 1. 2019
The Easter Christmas Release (this one is somewhat late).
Python:
- We are excited to introduce the SimGrid/S4U interface to your neighborhood
- Not complete yet: asynchronous activities (amongst others) are still missing
- Still ongoing: the interface may change in the future. We need more
testers! Please report any glitches.
- No new project using Java should start now. Please switch to Python.
General:
- Some of the internal cleanups may lead to speed improvements:
- The hard limitation on the amount of simulated actors with
Java+Mac was removed. Now, the available memory is the only limit.
- Our refcounting was tidyied, leading to 10% speedups in some cases.
- We are still working on making our code robust to the actor kills
and hosts' churn. Things are improving, but it's not perfect yet.
- Replaced our own code to display a backtrace (that was forking addr2line)
with the Boost.Stacktrace library.
You won't see your backtraces without this optional dependency.
- Bump cmake dependency to 3.5 (provided by Ubuntu 16.04).
- Stop setting random seed with srand() at initialization.
XML
- In <host> and <peer>, 'availability_file' is now 'speed_file'.
XML file version remains 4.2 since old files are still compatible.
Java:
- Process termination which was broken at version 3.21 has been repaired.
- Expose host load plugin: loadInit, getCurrentLoad, getComputedFlops, getAvgLoad
- Hide the examples into examples/deprecated. New users should use Python.
MSG:
- Drop MSG_process_create_from_stdfunc() from the API.
This C++ function was a pimple in the C API, made necessary at some
point by the Java bindings. This is fixed now.
- Hide the examples into examples/deprecated. New users should use S4U.
- MSG_process_create and MSG_process_attach now crash if the host used
to run this process is off.
- Fix the protype of MSG_process_on_exit()
Now use (int,void*) callbacks instead of (void*,void*) ones.
The implementation was ways too messy to actually work, I guess.
SMPI:
- Change the way SMPI_SAMPLE_* macros work to avoid invalidating cache
too often and provide more accurate timings.
- Add -gdb, -lldb, and -vgdb shortcuts to help debug MPI codes with smpirun
- MPI_Alltoallw support
- Partial MPI nonblocking collectives implementation: MPI_Ibcast, MPI_Ibarrier,
MPI_Iallgather, MPI_Iallgatherv, MPI_Ialltoall, MPI_Ialltoallv, MPI_Igather,
MPI_Igatherv, MPI_Iscatter, MPI_Iscatterv, MPI_Ialltoallw.
- MPI_Request_get_status, MPI_Status_set_cancelled, MPI_Status_set_elements
support
- Basic implementation of generalized requests (SMPI doesn't
allow MPI_THREAD_MULTIPLE) : MPI_Grequest_complete, MPI_Grequest_start
XBT:
- Drop sg_cmdline. Please use xbt_cmdline instead.
- Drop the C xbt_os_mutex_t; Use the C++11 std::mutex.
- Drop the C xbt_os_sem_t; Use the C++ xbt::OsSemaphore.
OsSemaphore is implemented in a portable way with C++11 threads.
This should allow much more threads to be created at the same time,
allowing Mac and Java users to simulate many more actors.
- Implement the 'thread' factory with std::thread instead of xbt ones.
It is not possible to set the stack size with threads anymore, but
-fsplit-stack is the way to go nowadays when using threads.
- Drop the xbt_os_thread_t module (now unused)
- Drop xbt_ex_display(), use simgrid::xbt::log_exception() instead.
- Drop xbt_str_join_array().
- Drop cunit, use Catch2 instead.
Kernel:
- Many cleanups in the kernel::activity namespace. This was long
overdue, and shall open the path to many future endeavors.
Fixed bugs:
- #132: Java: a process can not shut down its own host
- #220: S4U: detached send causes exception if sender terminates soon after sending
- #261: Document the parameters of parallel execution's constructor
- #300: [s4u] BarrierPtr is missing
- #314: SMPI args internal cleanup
- #316: Fix a bug related to the CPU utilization of multi-core VM
- #318: Invalid trace file when using option --cfg=tracing/smpi/display-sizes:yes
- #324: S4U: Attempting to create an actor on turned off host segfaults instead of throwing
- #325: Turning off a host has different behavior on sleeping actors and computing actors
----------------------------------------------------------------------------
SimGrid (3.21) October 3. 2018
The Restarting Documentation (TRD) Release.
Documentation:
- Convert the user manual to Sphinx for improved usability.
Unstable version now hosted on https://simgrid.frama.io/simgrid
Many glitches remain; stay tuned.
- Restrict the Doxygen reference API to the installed header files.
(documenting all internals this way is overkill + doxygen scales badly)
- New tutorial on S4U (highly inspired from the old MSG one, but with
a git repository to fork as a starting point, and a docker image)
- Started but not finished a SMPI tutorial inspired from the JLPC'14 one.
- The Developper manual is still to be converted (not compiled until then)
- Some parts are still missing in this conversion (such as the
platform and deployment sections) while others were blindly converted
and would need more love (such as the configuration flags).
Things will certainly further improve in the future releases.
S4U new features:
- Fully reimplement auto_restart mechanism that was utterly broken
(fix #22, #131 and #281 that were all very related)
- Implement semaphores (fix #303 and #291)
- When creating an actor from a function and its parameters,
move-only parameters are not allowed anymore, as it would prevent
the actor to be restartable if its parameters are consumed on run.
- s4u::Io: IOs go asynchronous as activities. New methods in s4u::Storage:
- io_init(sg_size_t, s4u::Io::OpType) to create a READ or WRITE asynchronous
IO operations that can be started, waited for, or canceled as a regular
activity.
- read_async(sg_size_t) and write_async(sg_size_t) which are wrappers on
io_init() + start()
Tracing:
- Rename 'power' and 'power_used' variables into 'speed' and 'speed_used'
- New host variable: 'core_count'
XBT:
- Remove xbt_os_thread_specific features
- Remove portability wrapper to condition variables
- Remove xbt_os_thread_yield()
SMPI:
- MPICH collective selector now mimics MPICH 3.3b
- OpenMPI collective selector now mimics OpenMPI 3.1.2 (default "tuned" setting)
- MPI_Init(NULL,NULL) is now allowed
- smpi/wtime option now injects time in gettimeofday or clock_gettime calls
- Command-line arguments should now be handled properly in Fortran simulations
- MPI Attributes and related callbacks should work in Fortran
- Apps using C + Fortran should now work
- MPI_* calls now check for non MPI_SUCCESS return values and emit warnings
- Support MPI_Error_String
Java:
- Due to an internal bug, Msg.run() must now be your last line.
We hope to fix it in a future release, and we are sorry for the inconvenience.
Fixed bugs:
- #22: Process autorestart seem to only work with CAS01 cpus
- #56: Feature request: dynamic SMPI replay
- #93: simgrid should not eat --help
- #111: How to change the loopback link characteristics?
- #116: Platform cluster radical
- #117: Platform cluster/host order
- #129: Replay_multiple on steroid
- #131: Java : autostart does not work
- #143: Setting a breakpoint at a given time
- #235: Network energy models should be integrated
- #264: Add ptask L07 resource tracing
- #271: Dynamic and manual replay of SMPI traces
- #279: Breakpoints option uses time deltas instead of absolute time
- #281: Daemonized actor no longer daemonized after an auto-restart
- #282: TIT convertor erase traces when using absolute path in the trace list file
- #285: segfault when a s4u actor kills itself with msg process tracing activated
- #286: Pajé traces are not exposing the number of cores
- #287: Command-line parsing should stop when encountering '--'
- #288: MPI_Init(NULL, NULL)
- #289: Improve documentation of Actor::on_destruction and this_actor::on_exit()
- #290: Method to check if a mailbox has a message ready to be consumed
- #291: [s4u] semaphores do not seem available yet
- #294: Alltoall collective implementation of mpich
- #295: Starting a Host as OFF is not supported
- #296: DTD too permissive
- #299: Add a s4u barrier example
----------------------------------------------------------------------------
SimGrid (3.20) Released June 24. 2018
The proxy snake_case() release.
S4U:
- The whole API is now using snake_case() instead of mixing it with camelCase.
Compatibility wrappers in place for one year.
- We took the opportunity to increase the consistency of the API.
Please report any remaining glitches.
- New function simgrid::s4u::Host::get_actor_count:
Returns the number of actors running on a specific host.
SMPI:
- SMPI is now tested with ~45 proxy apps from various sources, with none or
only minor patching needed: check https://github.com/simgrid/SMPI-proxy-apps
- Replay: The replay file has been re-written in C++.
- Replay: Tags used for messages sent via MPI_Send / MPI_Recv are now
supported. They are stored in the trace and used when replayed.
- Basic support of MPI_Cancel. Robustness not guaranteed.
- Support of MPI_Win_allocate_shared, MPI_Win_shared_query, MPI_Comm_split_type
(only for MPI_COMM_TYPE_SHARED).
- New option: smpi/privatize-libs, to add external shared libs to be privatized
by SMPI. They will be copied locally and loaded separately by each process.
Example --cfg=smpi/privatize-libs:"libgfortran.so.3;libscalapack.so".
- Tracing: add tracing for MPI_Start, Startall, Testall, Testany
- Interception of getopt, getopt_long and getopt_long_only calls to avoid issues
with internal index optind with multiple processes. Only works if MPI_Init has
already been called.
- Fortran: SMPI builds a mpi.mod file which should allow use of "use mpi"
syntax without preprocessing tricks.
TRACE
- Change --cfg=tracing/msg/vm to --cfg=tracing/vm as virtual machine
behavior tracing is no longer limited to MSG
- TIT (Time Independent Traces): We finally support tags. Unfortunately,
this means that traces now need to be updated or re-obtained; both Irecv
and Isend lines in your traces have now in total 5 mandatory fields:
<rankid> <command> <to/from rankid> <tag> <size>
To update your traces, it suffices to add a 0 for the tag here.
- TIT now also supports waiting for a distinct request via MPI_Wait.
Wait/Test now wait for a specific request, not just the last one that was
issued. This unfortunately means another update, because we need to
identify which request you want to wait for. We do this via the
triplet (sender, receiver, tag), which needs to be added:
<rankid> <command> <sender> <receiver> <tag>
- Actions are now in lower case (e.g., "allReduce" becomes "allreduce").
New script simgrid_convert_TI_traces.py: upgrade your TI traces.
MSG
- The deprecation of MSG is ongoing (but this should not impact you).
Many MSG functions are now simple wrappers on the C API of S4U. If
you wish to convert your code to S4U, find the S4U counterparts of
your MSG calls in src/msg/msg_legacy.cpp.
- MSG can still be used, but won't evolve anymore.
Plugins:
- Allow to run the Link energy plugin from the command line with
--cfg=plugin:link_energy
- Rename Energy plugin into host_energy
- Rename Load plugin into host_load
simix:
- Add parameter --cfg=simix/breakpoint to raise a SIGTRAP at given time.
- kill simix::onDeadlock() that was somewhat dupplicating s4u::on_deadlock()
- Improve performance when handling timeouts of simix synchros.
XBT:
- Config: the C API is now deprecated (will be removed in 3.23), and
the C++ API has been slightly improved.
Other:
- Fix several build issues on OSX.
- Move simgrid_config.h to simgrid/config.h (old header still working)
Fixed bugs:
- #143: Setting a breakpoint at a given time
- #150: Inconsistent event names in SMPI replay
- #258: daemonized actors hang after all non-daemonized actors have completed
- #267: Linker error on unit_tmgr
- #269: SMPI: tracing of MPI_Wait/all/any broken
- SMPI: Fix various crashes with combined use of MPI_PROC_NULL and MPI_IGNORE_STATUS
----------------------------------------------------------------------------
SimGrid (3.19.1) Released March 22 2018
The Fixed ABI Release.
As you may know, we are currently refactoring SimGrid in deep.
Upcoming SimGrid4 will be really different from SimGrid3: modular,
standard and extensible vs. layered, homegrown and rigid. C++ vs. C.
Our goal is to smooth this transition, with backward compatibility
and automatic update paths, while still progressing toward SimGrid4.
SimGrid remains open during works: The last pure SimGrid3 release was
v3.12 while all subsequent versions are usable alpha versions of
SimGrid4: Existing interfaces remain unchanged, but the new S4U
interface is budding and the internals are deeply reorganized.
Since 2015, we work hard to reduce the changes to public APIs. When
we need to rename a public library symbol in S4U, we let your
compiler issue an explicative warning when you use the deprecated
function. These messages remain for four releases, i.e. for one full
year, before turning into an error. Starting with v3.15, your can
also adapt to API changes with the SIMGRID_VERSION macro, that is
defined to 31500 for v3.15, to 31901 for v3.19.1 and so on.
Starting with this v3.19.1, our commitment to reduce the changes to
the public interfaces is extended from the API to the ABI: a program
using only MSG or SimDag and compiled against a given version of
simgrid can probably be used with a later version of SimGrid without
recompilation. We will do our best... but don't expect too much of
it, that's a really difficult goal during such profund refactoring.
The difference between v3.19 and v3.19.1 is that the former was
accidentally breaking the ABI of MSG, while the later is restoring
the previous ABI.
S4U and kernel APIs will still evolve until SimGrid4, with one-year
deprecation warnings as currently. In fact, cleaning up these
interfaces and converting them to snake_case() is one release goal of
v3.20. But don't worry, we are working to smooth this upgrade path.
In summary, new projects should start with S4U to benefit of the
future, but old MSG projects should still be usable with no change.
----------------------------------------------------------------------------
SimGrid (3.19) Released March 20 2018
The Moscovitly-cold Spring release.
SMPI user-visible improvements:
- Support of MPI_Type_create_subarray
- Unimplemented MPI calls now abort (instead of issuing a warning)
Some harmless calls around errhandlers still only issue a warning.
- MPI_Sendrecv was behaving badly when MPI_PROC_NULL was sender or receiver.
- Starting MPI ranks (or even MSG/S4U actors) once the simulation started
is now possible (mmap privatization was refactored to this end).
Adding new SMPI instances should also be technically doable now.
SMPI internal cleanups:
- Removed index notion from SMPI -- no more "getPid() - 1"!
In tracing files, actors are called rank-<PROCESS-ID> but these are
NOT the real ranks (tracing several communicators is not supported yet).
- Deprecate smpi_process_index() (will be removed in 3.22)
- Better testing on classical proxy apps, through an external project:
https://github.com/simgrid/SMPI-proxy-apps/
Documentation
- Use a graphical TOC to make it easier to find the documentation you need
- Revamp the MSG tutorial
MSG
- Fix MSG_task_get_remaining_work_ratio(): return 1.0 for unstarted tasks.
- Remove parameter of MSG_process_killall().
Resetting the PID was bogus anyway (several actors could have the same PID).
S4U
- Execution->setHost() can be called after start(), to migrate it.
- Comm::test_any() is now implemented.
- s4u::Actor now has onCreation() and onDestruction() signals.
- Install some kernel header files for the users' plugins and more.
Warning, their API is really not stable yet.
include/simgrid/kernel/resource/{Action,Resource,Model}.hpp
include/simgrid/kernel/routing/*Zone.hpp
SimDag
- SD_task_dependency_add(): remove unused parameters 'name' and 'data'.
Build System and other cleanups:
- Remove unused run-time parameter "tracing/onelink-only".
- SimGrid now works with intel compilers (MC still buggy)
- Remove XBT_LOG_CONNECT, it should be useless nowadays.
- tesh kills subprocesses on timeout (but not on windows)
PLATFORM:
- Rename FULLDUPLEX into SPLITDUPLEX (old name still accepted)
Fixed bugs:
- #194: Feature request: simgrid::s4u::Comm::test_any()
- #245: migrating an actor does not migrate its execution
- #253: Feature Request: expose clusters as objects
- #254: Something seems wrong with s4u::Actor::kill(aid_t)
- #255: Tesh broken on Windows
- #256: Modernize FindSimGrid.cmake
- #257: Fix (ab)use of CMake install
----------------------------------------------------------------------------
SimGrid (3.18) Released December 24 2017
The "Ho Ho Ho! SimGrid 4 beta is coming to town" release.
MSG IS NOW DEPRECATED. Please use S4U for new projects.
- Support for MSG should not be removed from SimGrid before 2020,
but future evolutions will be limited to the integration of
user-provided patches.
S4U New features
- s4u::onDeadlock() signal, run before stopping the simulation.
- s4u::Exec: asynchronous executions (abstraction of background
threads)
- s4u::Host->getLoad() returns the achieved speed in flops/s
S4U API changes
- Link::name() is deprecated and will be removed in v3.21, use
Link::getCname() instead.
- Mailbox::getName() changed to return a std::string, use
Mailbox::getCname() instead to get a char*.
- Storage::getName() changed to return a std::string, use
Storage::getCname() instead to get a char*.
- s4u::allStorages() becomes s4u::getStorageList(whereTo)
to have both allocation and deallocation of the map in user space.
MSG
- Forbid the use of MSG_task_get_flops_amount() on parallel tasks:
The amount of flops remaining to do is a vector, not a scalar.
- Introduce MSG_task_get_remaining_work_ratio(), which does what its
name implies on both sequential and parallel tasks.
- Both changes fix GitHub's #223 using PR #237 as a basis.
Thanks Michael Mercier.
- Most examples were converted to S4U and hidden elsewhere at tests.
Rationale: we still want MSG to work; we want newcomers to use S4U.
SURF
- LMM stuff moved to its own namespace: simgrid::kernel::lmm.
- Renamed LMM classes (e.g. s_lmm_system_t -> System).
SMPI
- Switch to the faster dlopen privatization mechanism by default
- Documentation improvments
Virtual Machines
- Live migration is getting moved to a plugin. Dirty page tracking is
the first part of this plugin. This imply that VM migration is now
only possible if one this function is called:
- C/MSG: MSG_vm_live_migration_plugin_init()
- C/C++: sg_vm_live_migration_plugin_init()
- Java: Msg.liveMigrationInit()
For C and C++, "simgrid/plugins/live_migration.h" also has to be
included.
XBT
- Define class simgrid::xbt::Path to manage file names.
- Removed unused functions:
- xbt/file.h: xbt_basename(), xbt_dirname(), xbt_getline()
- xbt/graph.h: xbt_graph_edge_get_length(), xbt_graph_edge_set_length,
xbt_graph_export_graphviz()
- xbt/str.h: xbt_str_join()
- Remove unused datatypes:
- xbt/heap.h: use std::priority_queue or boost::heap instead
- xbt/swag.h: use boost::intrusive::list instead
PLUGINS:
- New link_energy plugin for the consumption of the links.
- All of the operations on files and storage contents have been
packaged into a plugin (src/plugins/file_system). The current
public interface can be found in
include/simgrid/plugins/file_system.h
To use these functions you now have to initialize the plugin by
calling MSG_storage_file_system_init() just after calling
MSG_init() or sg_storage_file_system_init() just after creating
the Engine.
XML
- Remove the undocumented/untested tag <include>
TRACE
- Remove viva specific tracing as the tool is no longer maintained
Fixed bugs:
- #248: Kill of finished processes leads to segfault
- #240: xbt_cond_wait_timeout should gracefully return for C apps
- #239: please implement signal s4u::onDeadlock()
- #230: segfaults when exit() without run()
- #225: s4u::Actor::kill() doesn not really kill victims in a join()
- #223: MSG_task_get_flops_amount() not working with parallel tasks
- #222: Actor::kill() doesn't really kill and segfaults
- #221: odd LMM warning when killing an actor
- #120: Memory leak when the processes are forcefully killed
----------------------------------------------------------------------------
SimGrid (3.17) Released October 8 2017
The Drained Leaks release: (almost) no known leaks despite the tests.
Even very long simulations will preserve your memory: our testsuite
stresses SimGrid for over 45mn on fast machines for a coverage of
over 80%, and there is only one single known leak, of about 4kb.
S4U
- Comm.detach(): start and forget about asynchronous emission. A cleanup
handler may be given to free resources if the comm cannot be completed.
- this_actor::send(mailbox) is now mailbox->put()
- New: simgrid::s4u::Comm::wait_all()
- New: Host.execute() for remote executions.
SURF
- Unused option network/sender-gap is removed.
MSG
- Deprecate MSG_task_isend_with_matching(): This unused feature
really complicates our internals. Will be removed in v3.20.
Simix
- Improved context termination. It is now possible to free resources, even
when a process is forcibly killed.
- ContextBoost: add support for Boost versions above 1.61.
XBT
- Bring back run-time option --cfg=exception/cutpath to remove exception
backtraces.
- Removed unused functions:
- xbt/str.h: xbt_str_split_str(), xbt_str_subst(), xbt_str_ltrim(),
xbt_str_rtrim(), xbt_str_trim().
- xbt/xbt_os_thread.h: xbt_os_thread_cancel(), xbt_os_thread_detach().
Misc
- Removed header files obsolete since SimGrid 3.12:
msg/datatypes.h, msg/msg.h, simdag/datatypes.h, simdag/simdag.h.
- Fix many bugs:
- #3: SD_exit should be made optional
- #120: Memory leak when the processes are forcefully killed
- #159: Threading test regression in Actor refcounting
- #170: simgrid::s4u::Comm::wait_any() returns too many comms
- #185: simgrid::s4u::Engine::instance()->shutdown() segfaults
- #186: Actor::killAll() segfaults if some process is blocked on wait()
- #191: VM migration and pstate
- #192: Updating the energy consumptions of all hosts crashes with VMs
- #195: All actors have PID=0 in the logs
- #204: Sometimes segfault with thread contexts and mmap privatization
- #222: Actor::kill() doesn't really kill and segfaults
- #225: Actor::kill() doesn't really kill when victims are doing a join()
----------------------------------------------------------------------------
SimGrid (3.16) Released June 22. 2017.
The Blooming Spring Release: developments are budding.
Portability status:
- FreeBSD: Disable SMPI mmap privatization, switch automatically to dlopen.
- Mac, BSD: dlopen+thread broken, switch automatically to raw contexts.
- Java JAR file should be finally fully working on Mac OSX too.
XML platforms: Switch to platform v4.1 format.
* This is (mainly) a backward compatible change: v4 are valid v4.1 files
- <zone> can be used as a synonym for the now deprecated <as>
- <zoneRoute> can be used as a synonym for the now deprecated <asroute>
- <bypassZoneRoute> an be used as a synonym for the now deprecated <bypassAsRoute>
- <actor> can be used as a synonym for the now deprecated <process>
- state_file and avail_file periodicity is now easier to express
(check the documentation) the old behavior should still work.
* Storage: not backward compatible, but it seems that nobody ever used it
- Remove attribute 'content_type' of <storage_type>: was never used
- Make attribute 'model' of <storage_type> optional: for future usage
- Remove Bconnection model property: was never (in)validated. Replaced by
the maximum of the read and write bandwidth as a resource constraint.
SimDag
- New and Backwards Compatibility break:
SD_simulate_with_update (double how_long, xbt_dynar_t changed_tasks_dynar)
When one wants to get the list of tasks whose states have changed during a
simulation round, s/he has to allocate and free a dynar and use it as
argument to this function. The former SD_simulate (double how_long)
now returns void.
Virtual Machines
- Allow multicore VMs, with the correct sharing (unless you overcommit)
BUG: vCPU overcommitting still leads to buggy sharing, though. WIP.
Energy
- New function to update the consumption of all hosts at once.
- Fix the model for multi-core hosts, linear in the amount of busy
cores with an abnormality for pIdle (see paper).
BUG: Still not satisfactory for ptask on mono-cores.
MSG
- The netzone are now available from the MSG API.
The old names still work, but are now deprecated.
SMPI
- New algorithm to privatize globals: dlopen, with dynamic loading tricks
- New option: smpi/keep-temps to not cleanup temp files
- New option: smpi/shared-malloc-blocksize . Relevant only when global shared
mallocs mode is used, allows to change the size of the fake file used
(default 1MB), to potentially limit the number of mappings for large runs.
- Support for sparse privatized malloc with SMPI_PARTIAL_SHARED_MALLOC()
- Fortran ifort and flang compilers support
- New RMA calls supported (experimental):
- MPI_Win_allocate, MPI_Win_create_dynamic, MPI_Win_attach
- MPI_Win_detach, MPI_Win_set_info, MPI_Win_get_info
- MPI_Win_lock_all, MPI_Win_unlock_all, MPI_Win_flush
- MPI_Win_flush_local, MPI_Win_flush_all, MPI_Win_flush_local_all
- MPI_Op_commutative, MPI_Fetch_and_Op, MPI_Compare_and_swap
- MPI_Rput, MPI_Rget, MPI_Raccumulate, MPI_Rget_accumulate
S4U
- New: this_actor::isMaestro() returns whether we are in kernel mode.
- Behind the scene, ActivityImpl are now automatically refcounted.
Removing this nasty bottleneck will greatly simplify our internals.
XBT
- Replay: New function xbt_replay_action_get():
Retrieve the function previously associated to an event type.
- DROPPED FUNCTION: xbt_str_varsubst()
- DROPPED MODULE: strbuff. We don't need it anymore.
- DROPPED MODULE: matrix. We don't need it anymore.
- DROPPED MODULE: lib. We don't need it anymore.
-- Thu Jun 22 22:48:12 CEST 2017 -- Da SimGrid team <simgrid-devel@lists.gforge.inria.fr>
SimGrid (3.15) Released March 22. 2017
The Spring Release: continuous integration servers become green
We fixed even the transient bugs on all target architectures:
Linux (CentOS, Debian, Ubuntu, Fedora), Mac OSX (Mavericks, El Capitan)
Windows, FreeBSD, NetBSD.
- Rename NetCards to NetPoints.
This was intended to help NS3 users, but that's not a netcard.
That's a point in the routing algorithm, let's avoid wrong simplifications.
MSG
- New: MSG_process_yield(). Stop and yield to other processes.
- New: MSG_process_daemon(). Daemon processes are automatically killed
when the last non-daemon process terminates
- New: MSG_process_ref/unref(). Fiddle with the process refcounting.
- Renamed MSG_energy_plugin_init() -> MSG_host_energy_plugin_init()
to make room for the upcoming network energy plugin.
- Drop MSG_host_get_current_power_peak: duplicates MSG_host_get_speed
Java
- Ensure that an actor can kill itself with Process::exit()
- Kill the obscure NativeException. Nobody want to survive the issues
it denotes, so use JniException that is a RuntimeException (not to
be caught explicitly).
- Partial bug fix in initialization. SimGrid flags on command line were
consumed at C level but stayed in the original Java String[] args.
This could mess users' args[i] if SG flags were not put at the end of
the command line.
The SimGrid flags are now removed from the Java arguments. However,
the number of arguments REMAINS UNCHANGED. It is then UNSAFE to test
if args.length is greater than the number of YOUR OWN ARGUMENTS.
It might be if you have --log or --cfg flags in the command line.
- Fix numerous memleaks all around the place. In particular, around VMs.
S4U
- New callbacks:
- s4u::onPlatformCreated: right before the simulation starts
- s4u::onSimulationEnd: right after the main simulation loop
- s4u::onTimeAdvance: right after a clock change (time is discrete)
- s4u::Host::onSpeedChange: when the pstate is changed, or when an
event from the availability_file changes the avail speed.
- Links are now usable from s4u
- New: Engine::hostList() and Engine::hostCount(). Still clumsy.
- New: Actor::suspend(), Actor::resume(), and Actor::migrate(new_host)
- New examples: The conversion of MSG examples to S4U has begun
- Actors: create, kill, migration, and suspend
- Applications: master-worker and token-ring
- Action replay: communications and storage
- Drop Host::getPstateSpeedCurrent() which duplicates Host::speed()
SimDag
- Backwards Compatibility breaks
- The SD_route_get_list and SD_route_get_size functions have been removed.
They are replaced by the unique
void sg_host_route(sg_host_t src, sg_host_t dst, xbt_dynar_t links)
The route from src to dst is built in the links dynar whose size can
be obtained with xbt_dynar_length.
- The SD_route_bandwidth and SD_route_latency functions have been replaced by
sg_host_route_bandwidth and sg_host_route_latency.
Macros ensure the backwards compatibility, but you should fix your code
SMPI
- Major C++ rewrite ongoing (SMPI used to be C compiled in C++).
This can break codes using SMPI internals (from private.h instead of the public smpi.h).
- Bump our claim of support from MPI 1.1 to MPI 2.2.
We don't support 100% of it, but it should be enough. Contact us if not.
- MPI_Win_lock/unlock, MPI_Get_accumulate support added (as for all RMA, implementation is
naive and probably inaccurate)
- New algorithm for SMPI_SHARED_MALLOC: global, where all blocks are
mapped onto a unique small file using some system magic.
- Bugfix: smpirun was sometimes misusing hostfile when wrongly terminated
- Fortran: cleanups, fixes, support of user-added operations
- RMA: MPI_Accumulate are applied in correct order.
- RMA: MPI_Win_{complete/post} shouldn't miss messages anymore.
- Better support for MPI_IN_PLACE
- Support for MPI_Win attrs and keyvals.
- Support MPI_Comm_group_create, MPI_Type_size_x
- MPI ops should be properly applied to all allowed types
- Replace implementation of MPI_Bcast "scatter followed by rdb allgather"
algorithm by a non failing one from mpich.
XBT
- Kill the fifo data container: we don't use it anymore.
XML
- A <cluster> can now be created with different speed values to represent pstates.
see examples/platforms/energy_cluster.xml for an example.
SimGrid (3.14.159) Released December 28. 2016
The previous release was only a crude approximation of a Pi release;
this one should be a bit better.
- Revert a last minute change that broke on Mac OSX.
- Fix the changelog and NEWS files.
- Improve the documentation, in particular of the routing module.
- Integrate some patches flying around in Debian.
SimGrid (3.14) Released December 24. 2016
Documentation
* The whole documentation was reworked and reorganized. There is
still much room for improvement here, but we're on it.
Infrastructure
* We now need python3 for our testing infrastructure.
* Model checking needs libevent
MC
* Now works on FreeBSD
SMPI
* Privatization now works on FreeBSD
* Privatization is now activated by default in smpirun
You can turn it off with -no-privatize if something goes wrong.
* Call-location tracing for SMPI.
You can add the exact location (filename / linenumber) of an MPI call to
your trace files and slow down or speed up the simulation between two
consecutive calls by using an adjustment file (see the documentation).
* Fixed computation of timings for MPI_Send, MPI_Recv & possibly also others
We've found a bug that prevented SMPI to account for MPI_Send, MPI_Recv
and others (in some cases) in a correct way. That is, the smpi/os, smpi/or
values were ignored in some cases. The timings of these functions can now
be significantly different.
* smpi/cpu-threshold:-1 should become smpi/simulate-computation:no
smpi/running-power is renamed to smpi/host-speed
* smpi/grow-injected-times option to enable or disable multiplication of the
timings injected in MPI_Iprobe, or MPI_Test. Enabled by default, which can
make simulation less precise (but also much faster).
* smpirun script should be (much) faster for large deployments.
* SMPI tracing: fixed issue with poor matching of send/receives.
* Replay: Fix broken waitall
New functions and features
* MSG_parallel_task_execute_with_timeout, to timeout computations.
Dropped / renamed functions and features
* msg_mailbox_t and associated functions. Use s4u::Mailbox instead.
- MSG_mailbox_is_empty() -> Mailbox::empty()
- MSG_mailbox_front() -> Mailbox::front()
- MSG_mailbox_get_by_alias() -> simgrid::s4u::Mailbox::byName(name)
- MSG_mailbox_get_task_ext() -> MSG_task_receive_ext()
- MSG_mailbox_get_task_ext_bounded -> MSG_task_receive_ext_bounded
- MSG_host_(get/set)_params -> MSG_vm_(get/set)_params
* Don't pass the free_f parameter to property related functions:
- MSG_host_set_property_value()
- MSG_as_router_set_property_value()
- MSG_storage_set_property_value()
* VM properties. Since msg_vm_t are msg_host_t, just use
MSG_host_get_property_value() and friends
* VM I/O related things:
- Ignored parameter of vm_create: core_nb, disk_path and disk_size.
- Unimplemented save/restore methods
* MSG_as_router_get_property_value() was redundant with
MSG_environment_as_get_property_value().
- Removed MSG_as_router_*propert*() functions
- Added MSG_environment_as_set_property_value() for consistency
* xbt heterogeneous dictionnaries (created with xbt_dict_new()).
Well, they are still there for now, but deprecated with a warning.
Please switch to xbt_dict_new_homogeneous() before this is removed
for real.
* Task affinity. Its intended behavior (that was very badly tested
and probably not really working) was deceiving what most users
would have hoped here.
* xbt_os_sem_get_value: unused internally, deprecated on OS X El Capitan
* Option network/coordinates is now useless and should be dropped.
Storage:
* Added option storage/max_file_descriptors to allow more than 1024 files opened
SURF:
* Added option maxmin/concurrency_limit to allow more than 100 processes per host
* Added Dragonfly topology support
XML:
* Add Exa- and Peta- units such as EiB, EB, Eib, Eb for size, and
EiBps, EBps, Eibps, Ebps for bandwidth.
They may become useful to some lucky ones.
Java:
* New functions: msg.Comm.waitAll() and msg.Comm.waitAny()
* ex/app_tokenring: new example, very similar to the MSG Token Ring
* ex/async_waitAll: new example, on asynchronous communications
MSG:
* Memory usage should be decreased for simulations with a large number
of processes. This also helps for SMPI.
SimGrid (3.13) stable; urgency=low
The Half Release, a.k.a. the Zealous Easter Trim.
- We removed half of the lines, that were mostly experimental cruft.
v3.12 lasted 286000 lines of code, v3.13 is only 142000 lines
(not counting blanks and comments -- according to openhub.net)
- The internals are now compiled in C++ (and will soon be clean C++)
- We removed 75 klines of XML, 12 klines of Java, 5 klines of cmake,
59 klines of C, etc. We added only 29 klines of C++ in replacement.
* Backwards Compatibility breaks
- Removed Lua simulation bindings (switch to C or Java for that).
Lua can still be used to describe platforms
- Removed Java kernel plug-ins.
Will be reintroduced after the ongoing major internals reorg.
- In MSG
- the following functions were removed.
They were too specific and should be reimplemented in a generic
way, with filter function.
- MSG_task_listen_from_host
- MSG_mailbox_get_count_host_waiting_tasks
- MSG_mailbox_put_with_timeout was removed.
Please use MSG_task_send_with_timeout instead.
- In SimDag
- the SD_application_reinit function was removed. It has been a noop for a while.
- The ACCESS_MODE of SD_workstation has been removed. This feature was not really usable and should soon be
replaced by a more flexible mechanism.
- The following functions thus do not exist anymore
- SD_workstation_get_access_mode
- SD_workstation_set_access_mode
- SD_workstation_get_current_task
- Basic estimation functions have been removed but can easily be replaced
- SD_route_get_communication_time => SG_route_get_latency() + amount / SD_route_get_bandwidth()
- SD_workstation_get_computation_time => amount / sg_host_speed()
- In Java
- VM.setBound(int load) is now VM.setBound(double bound) to meet the MSG semantics.
Use VM.getSpeed()*load/100 for the legacy behavior.
- In CMake
- option enable_tracing was removed. It was not doing anything for a while.
- In the ModelChecker:
- the model-checker now ptraces the model-checked process which means
you cannot use a debugger on the latter anymore (we might make this
optional in the feature);
- removed soft-dirty page tracking;
- remove model-checked side snapshot management,
MC_snapshot() and MC_compare_snapshot();
- keep the MC_cut() function as a stub (it was not really working
in the previous release).
Options:
* All options are consistently in kebab-case. Old names are kept as alias.
XML platforms:
* Switch to platform v4 format.
- Rename from 'power' to 'speed' the attributes describing the amount of
flops that a <host>, <peer>, <cluster> or <cabinet> can deliver per second.
- In <trace_connect>, attribute kind="POWER" is now kind="SPEED".
- In <host> and <link>, attributes availability and state are gone.
It was redundant with state and availability traces, and with peak values.
- In <cluster>, attributes availability_file and state_file are gone.
It was too complex and unused.
- Kill <gpu>. Was not doing anything.
- The DOCTYPE points to the right URL:
http://simgrid.gforge.inria.fr/simgrid/simgrid.dtd
(the file at this address now documents the changelog since its v1)
- A warning is emitted for unit-less values (they are still accepted).
- speed. Default: 'f' or 'flops'. Also defined:
'Yf', 'Zf', 'Ef', 'Pf', 'Tf', 'Gf', 'Mf', 'kf'
'yottaflops', 'zettaflops', 'exaflops', 'petaflops', 'teraflops', 'gigaflops', 'megaflops', 'kiloflops'
- bandwidth. Default: 'Bps' bytes per second (or 'bps' for bits but 1 Bps = 8 bps)
Also defined in bytes: 'TiBps', 'GiBps', 'MiBps', 'KiBps', 'TBps', 'GBps', 'MBps', 'kBps', 'Bps'
And the same in bits: 'Tibps', 'Gibps', 'Mibps', 'Kibps', 'Tbps', 'Gbps', 'Mbps', 'kbps', 'bps'
- latency. Default: 's' second. Also defined:
'w' week, 'd' day, 'h' hour, 'm' minute, 'ms' millisecond, 'us' microsecond, 'ns' nanosecond, 'ps' picosecond
* bin/simgrid_update_xml can upgrade your files automatically (won't convert unit-less values)
tools/sg_xml_unit_converter.py may help (but it's just a warning and will probably ever be).
S4U
* s4u::Host is now the preferred public interface to the Host features.
sg_host_* functions are C bindings to the exact same behavior
MSG_host_* and SD_workstation_* are #define to the sg_host_* ones