The Granularity of Multiple Outputs

[gychen233]

I fully thank the organizers for addressing our comments, but I still have a lot of confusion.

As an extension of #20 #28.

I would like to clarify the definition of 'incompatible granularity ' when grouping operations.

Suppose two operations in a subgraph produce output matrices of sizes 256x256 and 128x256, respectively. Can I use an execution granularity of 128x128 and simply mask/silently ignore the execution when the spatial grid exceeds the boundaries of the smaller matrix?

Additionally, is it allowed for a single subgraph to have multiple output operations of different types—specifically, one being a MatMul and the other a Pointwise operation?

[yarongmu-google]

Thanks for the question.

• Suppose two operations in a subgraph produce output matrices of sizes 256x256 and 128x256, respectively. Can I use an execution granularity of 128x128 and simply mask/silently ignore the execution when the spatial grid exceeds the boundaries of the smaller matrix? -> Yes, this is valid. The execution granularity [w, h, k] sets the tile size, not the grid dimensions. Different operations in the same subgraph may have different tile counts based on their output sizes; each operation's output is fully tiled and computed. This is standard output-stationary scheduling, as shown in several PROBLEM.md examples.

• Additionally, is it allowed for a single subgraph to have multiple output operations of different types—specifically, one being a MatMul and the other a Pointwise operation? -> yes, as long as your scheduling produced all results at least once.

[yarongmu-google]

I will resolve this for now. Please open a new issue, reference this one, if the above doesn't make sense.

[yarongmu-google]

Update: given the wide range of confusions caused by multiple outputs from a single op, I have removed all such cases from the benchmarks (both released and unreleased). Now all ops can only have one single output. Hope this makes things easier.