BundleToResources adjustment and optimization

[echicoine-icf]

Changes:

• Retain folder structure to easily trace source of extracted resources
• Added new argument handling to delete bundles after operation completes: -db=true
• Added new BundleToResources test to account for new delete bundle argument
• Optimized code to handle large file loads

https://icf.atlassian.net/browse/CQF-2407

• I've read the contribution guidelines
• Code compiles without errors
• Tests are created / updated
• Documentation is created / updated

[codecov]

Codecov Report

Attention: Patch coverage is 58.21918% with 61 lines in your changes missing coverage. Please review.

Project coverage is 21.17%. Comparing base (9694fc0) to head (6b5410e).
Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
...encds/cqf/tooling/operation/BundleToResources.java	58.21%	40 Missing and 21 partials ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master     #543      +/-   ##
============================================
+ Coverage     21.00%   21.17%   +0.16%     
- Complexity     1656     1666      +10     
============================================
  Files           298      298              
  Lines         26121    26218      +97     
  Branches       4149     4168      +19     
============================================
+ Hits           5488     5552      +64     
- Misses        19737    19756      +19     
- Partials        896      910      +14     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[raleigh-g-thompson]

@echicoine-icf What is the issue you found with the ThreadUtils usage?

[echicoine-icf]

@echicoine-icf What is the issue you found with the ThreadUtils usage?

I found with larger operations, like refreshing the ecqm-content-qicore-2024 repo (located here https://github.com/cqframework/ecqm-content-qicore-2024) my mac seemed to be overloaded by the system-managed cached thread pool. In Windows, it works great (and when I wrote the class, I did so on my PC.)

I read up on it, turns out this is a common observation and it boils down to unix based operating systems trusting developers more in managing threads etc. Limiting the thread pool count prevents a unix system from being overburdened to the point of locking up and becoming unusable.

[raleigh-g-thompson]

Yeah that makes sense.

Your work on the bundles looks good but we need to do something a bit different with the threading changes.

There is a concern that the ThreadUtils class is used across other features and we haven't seen other users report issues. Nor has our team experienced these issues.

That being said, to move the PR forward here are 2 options

1. Move ThreadUtils changes to another PR
2. Add a flag to enable the fixed threading. If the fixed thread flag is passed in, with some thread count, then use the fixed model, else default to the cached model.

Using a flag will allow users that haven't experienced the same issues as you have won't experience and changes.

Even if option 1 is used we'll still want to see a flag to allow users to choice to override the cached model.

[echicoine-icf]

When I wrote the class it was to help with performance during the RefreshIG operation, but I also implemented it into ExtractMatBundle. So far these are the only two operations that utilize it so I won't be stepping on anyone's toes so to speak. As far as users reporting the issue, I think it's likely due to these operations not getting used much or getting used by Windows users. This work is to try to get out ahead of the user complaints. I do think it's a good idea thought to back out the change to this file and create a standalone PR for better tracking. I don't want to introduce any new flags because I feel like this should just work seamlessly for all users without them having to tweak anything.

[echicoine-icf]

new PR for ThreadUtils here: #555

[raleigh-g-thompson]

Thanks for pulling out the ThreadUtils change.