Add Support for Xcode 15 and iOS 17

[aircraft-cerier]

We are trying to use xctestrunner to run testing on iOS 17 devices with xcode 15.0

We are running into the follow errors when passing a command:
First when running the following command:
ios_test_runner --app_under_test_path ios-configurator.ipa --test_bundle_path ios-configurator-ui-tests.zip --work_dir ~/ios_test_runner_work_dir --test_type xcuitest test --id 00008020-001524340281002E

we get the following error regardless of what iOS our device is on:

Command line invocation:
    /Applications/Xcode15.0beta3.app/Contents/Developer/usr/bin/xcodebuild test-without-building -xctestrun ~/ios_test_runner_work_dir/TEST_ROOT/xctestrun.plist -destination id=00008020-001524340281002E -derivedDataPath /var/folders/6d/l0ggp9fj52g22q3x1ghl7bfh0000gq/T/tmp73pv379u -resultBundlePath /var/folders/6d/l0ggp9fj52g22q3x1ghl7bfh0000gq/T/tmp73pv379u/test.xcresult -collect-test-diagnostics=never

User defaults from command line:
    collect-test-diagnostics = never
    IDEBuildOperationResultBundlePath = /var/folders/6d/l0ggp9fj52g22q3x1ghl7bfh0000gq/T/tmp73pv379u/test.xcresult
    IDEDerivedDataPathOverride = /var/folders/6d/l0ggp9fj52g22q3x1ghl7bfh0000gq/T/tmp73pv379u
    IDEPackageSupportUseBuiltinSCM = YES

Writing result bundle at path:
	/var/folders/6d/l0ggp9fj52g22q3x1ghl7bfh0000gq/T/tmp73pv379u/test.xcresult

xcodebuild: error: Failed to build workspace temporary with scheme Transient Testing.: File is of an unsupported type; must be either an xctestrun file or xctestproducts bundle.

I am able to fix this issue by manually changing the name of the xctestrun.plist file to test.xctestrun and rerunning the command:
/Applications/Xcode15.0beta3.app/Contents/Developer/usr/bin/xcodebuild test-without-building -xctestrun ~/ios_test_runner_work_dir/TEST_ROOT/test.xctestrun -destination id=00008020-001524340281002E -derivedDataPath /var/folders/6d/l0ggp9fj52g22q3x1ghl7bfh0000gq/T/tmp73pv379u -resultBundlePath /var/folders/6d/l0ggp9fj52g22q3x1ghl7bfh0000gq/T/tmp73pv379u/test1.xcresult -collect-test-diagnostics=never

After doing this we run into another error:

Command line invocation:
   /Applications/Xcode15.0beta3.app/Contents/Developer/usr/bin/xcodebuild test-without-building -xctestrun ~/ios_test_runner_work_dir/TEST_ROOT/test.xctestrun -destination id=00008020-001524340281002E -derivedDataPath /var/folders/6d/l0ggp9fj52g22q3x1ghl7bfh0000gq/T/tmp73pv379u -resultBundlePath /var/folders/6d/l0ggp9fj52g22q3x1ghl7bfh0000gq/T/tmp73pv379u/test1.xcresult -collect-test-diagnostics=never

User defaults from command line:
    collect-test-diagnostics = never
    IDEBuildOperationResultBundlePath = /var/folders/6d/l0ggp9fj52g22q3x1ghl7bfh0000gq/T/tmpnp_tmxyg/test3.xcresult
    IDEDerivedDataPathOverride = /var/folders/6d/l0ggp9fj52g22q3x1ghl7bfh0000gq/T/tmpnp_tmxyg
    IDEPackageSupportUseBuiltinSCM = YES

Writing result bundle at path:
	/var/folders/6d/l0ggp9fj52g22q3x1ghl7bfh0000gq/T/tmpnp_tmxyg/test3.xcresult

Testing started
2023-07-06 16:21:39.428 xcodebuild[67560:380128] [MT] DVTDevice: Error locating DeviceSupport directory using Optional("arm64e") or Optional("arm64e"): nilError
2023-07-06 16:21:46.156 xcodebuild[67560:380128] [MT] IDETestOperationsObserverDebug: 11.913 elapsed -- Testing started completed.
2023-07-06 16:21:46.156 xcodebuild[67560:380128] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start
2023-07-06 16:21:46.156 xcodebuild[67560:380128] [MT] IDETestOperationsObserverDebug: 11.913 sec, +11.913 sec -- end

Test session results, code coverage, and logs:
	/var/folders/6d/l0ggp9fj52g22q3x1ghl7bfh0000gq/T/tmpnp_tmxyg/test3.xcresult

We do not run into this second error with iOS 16, only with iOS 17.

[tirodkar]

@me11kiy Mind looking into this. I think you might have hit this internally as well.

[me11kiy]

@aircraft-cerier, we've seen the issue with xctestrun file and fixed it with using xctestrun extension too. if you have a PR for that - please send it.
As for the second error - yes, I don't see iOS 17 device support under /Applications/Xcode_15.0.0_beta_4.app/Contents/Developer/Platforms/iPhoneOS.platform/DeviceSupport/. Unfortunately I don't currently have a real device with iOS 17. This discussion suggests it should just work. Are you seeing the same error with beta 4?

[keith]

should be fixed at HEAD, please test and report back!

[aircraft-cerier]

All good now, thanks!