Now that 4.1 is released, I can work on the Linux port once again

[felix91gr]

As per a weird Foundation bug in 4.0, we couldn't compile SKD on Linux. Now that 4.1 is out, that Foundation bug is also fixed. I should be able to go back to working on the port now. If anyone wants to help, feel free! Below is what's left to do:

State of the port

• Dependencies compile and pass tests on Linux
• SKD compiles on Linux
• SKD passes tests on Linux

The test suite actually crashes. My current guess is that it's due to some forced unwrappings done in SKD that may be non-Nil on Darwin, but Nil on Linux. We shall see.

After our tests pass, we should be almost done! Here's hoping we can get the port running before Swift 5.0 gets released ✊

[felix91gr]

If I’m right with the force-unwrapping issue, then the culprit might be Foundation. Here’s the current status of the Linux implementation. I’m gonna investigate what Foundation types we’re using. If they’re incomplete on Linux, that should be a red flag.

[felix91gr]

List of suspicious (or faulty) files

Foundation is incomplete on Linux, therefore I'm writing here all of the files that I find in SKD that use functionality that is either Mostly Complete, Incomplete or Unimplemented as per status page,

Suspicious

Here are the lines that use Mostly Complete parts of Foundation, with the name of the library they use:

• NSJSONSerialization in Completer/CompletionResult.swift
• Process in Project/Project.swift
• NSJSONSerialization in /Server/CompletionServer.swift

Surely Faulty

I couldn't find any use of Incomplete pieces of Foundation.

Definitely Faulty

I couldn't find any use of Unimplemented pieces of Foundation.

Annex: Clean Files

These are the files for which I couldn't find usage of anything that isn't marked as Complete in Foundation's Status Page. In other words, everything they use is marked as Complete, and therefore shouldn't be the source of any errors regarding Foundation.

• /Completer/Completer.swift
• /Extensions/Path+.swift
• /Project/ProjectType.swift

Files that apparently didn't need Foundation to begin with

These files still let the project compile after I erased their import Foundation line.

• /Extensions/Dictionary+.swift
• /Project/ProjectError.swift

[felix91gr]

Actually, before we do that, I found that this line in particular crashes. It might be unrelated to Foundation.

The crash is related to XCodeEdit. It's very weird.

[felix91gr]

Here's how it's crashing for me:

1. Have a clean directory:

felix@felix-X550LD ~/D/P/SKD> git clone https://github.com/terhechte/SourceKittenDaemon/ cleanSlate_1

2. Run make test:

Here's the expected output. The building procedure just works.

felix@felix-X550LD ~/D/P/SKD> cd cleanSlate_1/
felix@felix-X550LD ~/D/P/S/cleanSlate_1> make test
make: agvtool: Command not found
find: Arguments to -type should contain only one letter
FIXTURE_PATH="/home/felix/Documents/Programming/SKD/cleanSlate_1/Tests/SourceKittenDaemonTests/Fixtures" \
FIXTURE_PROJECT_DIR="/home/felix/Documents/Programming/SKD/cleanSlate_1/Tests/SourceKittenDaemonTests/Fixtures/Project" \
FIXTURE_PROJECT_FILE_PATH="/home/felix/Documents/Programming/SKD/cleanSlate_1/Tests/SourceKittenDaemonTests/Fixtures/Project/Fixture.xcodeproj" \
swift test
Fetching https://github.com/Carthage/Commandant.git
Fetching https://github.com/jpsim/SourceKitten.git
Fetching https://github.com/envoy/Embassy.git
Fetching https://github.com/felix91gr/XcodeEdit.git
Fetching https://github.com/felix91gr/FileSystemWatcher.git
Fetching https://github.com/antitypical/Result.git
Fetching https://github.com/drmohundro/SWXMLHash.git
Fetching https://github.com/jpsim/Yams.git
Fetching https://github.com/norio-nomura/Clang_C.git
Fetching https://github.com/norio-nomura/SourceKit.git
Fetching https://github.com/felix91gr/inotify.git
Cloning https://github.com/Carthage/Commandant.git
Resolving https://github.com/Carthage/Commandant.git at 0.12.0
Cloning https://github.com/felix91gr/XcodeEdit.git
Resolving https://github.com/felix91gr/XcodeEdit.git at 1.1.2
Cloning https://github.com/jpsim/Yams.git
Resolving https://github.com/jpsim/Yams.git at 0.4.1
Cloning https://github.com/norio-nomura/SourceKit.git
Resolving https://github.com/norio-nomura/SourceKit.git at 1.0.1
Cloning https://github.com/envoy/Embassy.git
Resolving https://github.com/envoy/Embassy.git at 4.0.2
Cloning https://github.com/felix91gr/inotify.git
Resolving https://github.com/felix91gr/inotify.git at 1.0.2
Cloning https://github.com/norio-nomura/Clang_C.git
Resolving https://github.com/norio-nomura/Clang_C.git at 1.0.2
Cloning https://github.com/drmohundro/SWXMLHash.git
Resolving https://github.com/drmohundro/SWXMLHash.git at 4.6.0
Cloning https://github.com/antitypical/Result.git
Resolving https://github.com/antitypical/Result.git at 3.2.4
Cloning https://github.com/felix91gr/FileSystemWatcher.git
Resolving https://github.com/felix91gr/FileSystemWatcher.git at 1.1.2
Cloning https://github.com/jpsim/SourceKitten.git
Resolving https://github.com/jpsim/SourceKitten.git at 0.18.4
Compile CYaml src/writer.c
Compile CYaml src/scanner.c
Compile CYaml src/reader.c
Compile CYaml src/parser.c
Compile CYaml src/emitter.c
Compile CYaml src/api.c
Compile inotify inotify_wrapper.c
Compile Swift Module 'XcodeEdit' (5 sources)
Compile Swift Module 'SWXMLHash' (3 sources)
Compile Swift Module 'Result' (2 sources)
Compile Swift Module 'Embassy' (29 sources)
Compile Swift Module 'FileSystemWatcher' (1 sources)

Then it gives me a bunch of warnings related to Swift 4.1...

/home/felix/Documents/Programming/SKD/cleanSlate_1/.build/checkouts/XcodeEdit.git--3656023722064191949/Sources/Extensions.swift:19:17: warning: 'flatMap' is deprecated: Please use compactMap(_:) for the case where closure returns an optional value
    return self.flatMap { $0 as? T }
                ^
/home/felix/Documents/Programming/SKD/cleanSlate_1/.build/checkouts/XcodeEdit.git--3656023722064191949/Sources/Extensions.swift:19:17: note: use 'compactMap(_:)' instead
    return self.flatMap { $0 as? T }
                ^~~~~~~
                compactMap
Compile Swift Module 'Yams' (15 sources)
Compile Swift Module 'Commandant' (10 sources)
/home/felix/Documents/Programming/SKD/cleanSlate_1/.build/checkouts/Embassy.git-6935331987155158314/Sources/TCPSocket.swift:221:49: warning: overlapping accesses to 'address', but modification requires exclusive access; consider copying to a local variable
        return try withUnsafeMutablePointer(to: &address) { pointer in
                                                ^~~~~~~~
/home/felix/Documents/Programming/SKD/cleanSlate_1/.build/checkouts/Embassy.git-6935331987155158314/Sources/TCPSocket.swift:231:34: note: conflicting access is here
            switch Int32(address.ss_family) {
                         ~~~~~~~~^~~~~~~~~
/home/felix/Documents/Programming/SKD/cleanSlate_1/.build/checkouts/Commandant.git-3714899713033202887/Sources/Commandant/ArgumentParser.swift:71:19: warning: 'characters' is deprecated: Please use String or Substring directly

And then it finishes compiling:

Compile Swift Module 'sourcekittend' (2 sources)
Compile Swift Module 'SourceKittenDaemonTests' (7 sources)
Linking ./.build/x86_64-unknown-linux/debug/sourcekittend
Compile Swift Module 'SourceKittenDaemonPackageTests' (1 sources)
Linking ./.build/x86_64-unknown-linux/debug/SourceKittenDaemonPackageTests.xctest

So far so good. But now is when it fails:

Test Suite 'All tests' started at 2018-04-02 19:02:54.140
Test Suite 'debug.xctest' started at 2018-04-02 19:02:54.294
Test Suite 'CompleterTests' started at 2018-04-02 19:02:54.294
Test Case 'CompleterTests.testCompletingAConstructor' started at 2018-04-02 19:02:54.294
[DEBUG] Checking setUp func in CompleterTests
[DEBUG] Starting super.setUp...
[DEBUG] Finished!. Creating a test var of type xcodeprojFixturePath...
[DEBUG] Worked!. Creating ProjectType from xcodeprojFixturePath...
[DEBUG] [EXTRA] Printing ProjectType
some(SourceKittenDaemon.ProjectType.project("/home/felix/Documents/Programming/SKD/cleanSlate_1/Tests/SourceKittenDaemonTests/Fixtures/Project/Fixture.xcodeproj"))
[DEBUG] [EXTRA] Success!
[DEBUG] ProjectType is in order. Poking project var...
[DEBUG] The error seems to be in the following line.
[DEBUG] It will fail on the 'public convenience init(propertyListData data: Data)' method,
[DEBUG] On XCProjectFile.swift
Fatal error: 'try!' expression unexpectedly raised an error: Data in .pbxproj file not in expected format: file /home/buildnode/jenkins/workspace/oss-swift-4.1-package-linux-ubuntu-16_04/swift/stdlib/public/core/ErrorType.swift, line 184
Current stack trace:
0    libswiftCore.so                    0x00007fe51cd3f220 _swift_stdlib_reportFatalErrorInFile + 221
1    libswiftCore.so                    0x00007fe51caae33c <unavailable> + 1368892
2    libswiftCore.so                    0x00007fe51cce7f12 <unavailable> + 3702546
3    libswiftCore.so                    0x00007fe51cce9379 <unavailable> + 3707769
4    libswiftCore.so                    0x00007fe51caada36 <unavailable> + 1366582
5    libswiftCore.so                    0x00007fe51cce7cdb <unavailable> + 3701979
6    libswiftCore.so                    0x00007fe51caada36 <unavailable> + 1366582
7    libswiftCore.so                    0x00007fe51cc1ae69 <unavailable> + 2862697
8    libswiftCore.so                    0x00007fe51caeb700 swift_unexpectedError + 318
Exited with signal code 4
Makefile:28: recipe for target 'test' failed
make: *** [test] Error 1

The [DEBUG] messages are debug prints that I put there. They only get compiled if we're on Linux.

XCProjectFile.swift is the one that makes the test suite crash. It's part of XCodeEdit (previously called XCode.Swift). I don't know why it crashes. This is the line that makes it crash

[felix91gr]

Regarding XCodeEdit / XCode.Swift:

1. They changed names when they hit v1.0
2. I was updating us to use it, and made a fork in the meantime because we had some API issues with the version that was then current
3. They have since released v2.0 and v2.1. Maybe they updated the API such that we can rely on them better.

I'll also ask about this crash. I don't understand what's the reason and they probably know.