Testing trait not behaving as expected

[DavidvanErkelens]

Hi there! I'm trying to build a test using the .container Suite macro, but it seems like when I run my tests in parallel the same instance of a dependency is used in multiple tests. I have a test like this:

import Factory
import FactoryTesting
import Testing

@Suite(.container)
struct MyTests {
    @Test()
    func testOne() async {
        let actorDouble = MyActorTestDouble()
        Container.shared.myActor.register { actorDouble }

        let viewModel = MyViewModel()

        await viewModel.run()

        #expect(await actorDouble.invoked == 1) // expectation failed, invoked == 2
    }

    @Test()
    func testTwo() async {
        let actorDouble = MyActorTestDouble()
        Container.shared.myActor.register { actorDouble }

        let viewModel = MyViewModel()

        await viewModel.run()

        #expect(await actorDouble.invoked == 1) // expectation failed, invoked == 0
    }
}

// Protocol for a background process
public protocol MyProtocol: Actor {
    func run()
}

// Actual implementation
public actor MyActor: MyProtocol {
    public func run() {
        // actual logic
    }
}

// Test double
public actor MyActorTestDouble: MyProtocol {
    public var invoked: Int = 0

    public func run() {
        invoked += 1
    }
}

extension Container {
    public var myActor: Factory<MyProtocol> {
        self { MyActor() }
    }
}

// View model invoking background process
@MainActor
class MyViewModel {
    @Injected(\.myActor) private var myActor: MyProtocol

    func run() async {
        await myActor.run()
    }
}

I would expect both tests to succeed here, but the only way I get them to succeed is either to run them individually or by adding .serialized to the @Suite macro. Is there anything I'm missing setting up these tests, or is the .container macro behaving differently than expected?

[alexgavrisco]

I'm running into a somewhat similar issue. Testing trait wasn't working for me (just crashing the test), so I just used a tiny wrapper that was doing the same thing as testing trait (creating a new container instance, setting it as task local and wrapping test body), but for individual tests.
It's working just fine as long as dependencies are not provided via @Injected, constructor injection works fine (I suppose because resolving is invoked on the same instance rather than Container.shared, which may be different).

[hmlongco]

Not sure this is the issue but as stated several times in the documentation, DO NOT IMPORT FACTORY INTO THE TEST SUITE, nor should you add it as a dependency to the test suite.

import Factory // NO NO NO
import FactoryTesting
import Testing

You can import FactoryTesting, but importing Factory itself will give you a duplicate version of the library and chaos will ensue.

[mbarringer]

You can import FactoryTesting, but importing Factory itself will give you a duplicate version of the library and chaos will ensue.

When we only import FactoryTesting, the compiler complains about how it Cannot find 'Container' in scope. For example, this will not compile:

import Testing
// import FactoryKit
import FactoryTesting
import Foundation

@Suite(.container)
struct TestExample {
    @Test func ExampleTest() {
        Container.shared.imageServiceProviding.register { MockImageService() }
    }
}

But uncomment import FactoryKit and the test compiles. Is it just import Factory that is expected to cause a problem? We're seeing the same behavior with the .container trait.

[sebastian-qm]

Hi @hmlongco ,
it would be really helpful if you could answer @mbarringer's last question:
"Is it just import Factory that is expected to cause a problem?"
Is it ok to import FactoryKit in the test suite or does it cause problems, too?

P.S. Thank you for your awesome work on this package!

[hmlongco]

@sebastian-qm You don't want to import Factory or FactoryKit. If you're using FactoryTesting you do want to import that into your test project. See the FactoryDemo.

[sebastian-qm]

@hmlongco Thank you for your quick reply.
I can't find any usage of FactoryTesting in the FactoryDemo (current main branch). Maybe I'm missing something?

And then the question how to use Container.shared in a test is still open: the test only compiles if FactoryKit is imported, as @mbarringer shows in his example #321 (comment) 🤔
Or should we avoid using Container.shared in a test that uses FactoryTesting?

[hmlongco]

I see the confusion. If needed you can "import FactoryKit" in the test code, just don't import the framework into the test project. FactoryKit in the app target, FactoryTesting only in the test target.

[sebastian-qm]

Thank you for the clarification @hmlongco !

A colleague of mine found this confusing snippet in the docs

.testTarget(name: "MyAppTests", dependencies: [
    "MyApp", 
    "Factory", 
    "FactoryTesting"
])

This is wrong, isn't it? According to what you said, "Factory" should not be in the dependencies array, right?

[Archdoog]

To clarify the discussion a bit. My issues were fully resolved when both of these steps were completed:

1. Upgraded my tests to use @Suite(.container) or @Test(.container).
2. Fully migrated my app from import Factory to import FactoryKit.

Once I migrated to FactoryKit, my tests typically had both imported.

import FactoryKit
import FactoryTesting

Before I did the migration, I was getting all kinds of concurrency errors in my tests based on the naming collision caused by import Factory