Remote block with tempVectors fails

[dionisiydk]

Transferring block with temp vectors logic does not work. And it can lead to some unexpected problems, infinite recursions or even image crash.
Simple example to reproduce is

| temp |
temp := 10.
remotePeer evaluate: [ temp := temp + 15  ].
temp.

Context should mark all tempVectors to be transferred by value.

[dionisiydk]

I created test for this in branch transferBlockWithTempVector

[LTangaF]

Switching from current master to the test branch plans to remove a lot of classes. I'm wondering what to put into the Metacello load to get closer to what your test branch originated from.

[LTangaF]

I suppose it probably just needs to be rebased. I'm not familiar with doing that in the context of an image, though. Do you have to re-issue the Metacello load after doing those sorts of git activities?

[dionisiydk]

It's because project was converted to Tonel. So all files were changed.
The branch only contains a test:

SeamlessRealCommunicationTests>>#testEvaluatingBlockWithTempManagedByVector
	| remotePeer result |
	remotePeer := self connectToServerPeer.
	"temp variable in following example is managed by temp vector 
	according to full block closure logic.
	Temp vectors should be forcibly transferred by value as integral part of context"
	self forkAndWait: [ | temp | 
		temp := 10.
		result := remotePeer evaluate: [ temp := temp + 5].
	]. 
	result should be: 15

[LTangaF]

I was thinking of your comment "Temp vectors should be forcibly transferred by value as integral part of context".
evaluate: doesn't have an evaluate:with: cousin to do value: calls with, so I'm guessing you mean we'd re-pack the block code and move the declaration and assignment of temp into the interior of the block, somehow determining that the block contains variables that are externally defined?

Through code re-packing, ending up with something that is instead effectively:

self forkAndWait: [
            result := remotePeer evaluate: [ | temp |
                                           temp := temp + 5 ] ]

[LTangaF]

So far, I've picked up the RBAssignmentNodes that are involved in the block interior using code like this:

| theblock temp assignmentNodes |
temp := 10.
theblock := [ temp := temp + 5 ].
"Get the temp variables that are actually referenced in the inner block"

assignmentNodes := theblock method ast body statements select: [ :thenode |
	thenode isAssignment and: [ (theblock asText findString: thenode variable name startingAt: 1) > 0 ] ].
assignmentNodes.

It's still a work in progress. I plan on injecting the tempVariables and assignmentNodes into the Block's compiled AST node's RBSequenceNode before sending it to remotePeer>>evaluate:

It's a work in progress, though, and I've got a question on the Development room of Discord about the the fact that the addNode messages don't seem to make any adjustments to the surrounding Node's assignment values, which are the pointers to their original locations in the original source.

[LTangaF]

It also just occurred to me, @dionisiydk, that what you meant by "forcibly transferred by value" could've been to establish BasysRemotePeer>>evaluate:with[:with[:with[:with]]] messages instead of the "defensive style" solution I'm working on.

I'm still new to SmallTalk programming, so it's possible that I've overthought this in that defensive direction. Probably my interpretation of "forcibly".

[dionisiydk]

Through code re-packing, ending up with something that is instead effectively:

self forkAndWait: [
            result := remotePeer evaluate: [ | temp |
                                           temp := temp + 5 ] ]

Semantically yet. It would lead to such equivalent code if we would pass temp vector by value.
It is required to fix a VM crash with more complex blocks. In current situation the remote side receives a corrupted block which state does not follow the VM expectation: the temp vector must be an Array instance but it is a proxy object.

From the other side the approach proposed here (pass by value) will break normal Smalltalk semantics: modification of the temp inside the block (on remote side) should be reflected on the home context (on client side) where the temp is declared and where it can be used after the return from the block.
A general solution should rewrite a block in a way where any remote temp writes would be encoded as explicit remote message (instead of simple store bytecode). My guess is that it is much harder to implement than what I proposed here.

I am not sure if you are aware what is a tempVector here. An example to understand:

| temp internalTempVector |
temp := #start.
internalTempVector := thisContext tempAt: 2.
[ temp := #end ].
internalTempVector  "==> #(#start)"

When temps written in the block are accessed from the outside of this block they are encoded with indirect arrays (a temp vector #(#start)).
Check this example without block:

| temp internalTempVector |
temp := #start.
internalTempVector := thisContext tempAt: 1.
internalTempVector "==> #start"

If we would know exactly indexes of tempVectors in the context the fix would be very trivial:

BlockClosure>>#prepareValueTransferBy: anObjectTransporter
        .........	
	outerContext tempVectors do: [:each |
             anObjectTransporter transferByValue: each] 
        ..........

[dionisiydk]

Also interesting that block rewrite would give us a solution for instance variables which is also broken now:

MyClass>>someDistrubutedMethod
        remotePeer evaluate: [myInstVar := #valueFromRemoteSide].

I think this task is super interesting to do but it could take a real time to implement.

[LTangaF]

I'm actually nearing completion on the block rewrite solution, but you threw me for a loop when you suggested that I may need to be able to reflect the value back into the original variable.

[LTangaF]

I guess my interpretation about this needing to be a defensive solution is correct, though.

[LTangaF]

Does the remote side have access to the local context (remote from it's perspective) ? We might be able to use communication via the context itself.

[dionisiydk]

I'm actually nearing completion on the block rewrite solution, but you threw me for a loop when you suggested that I may need to be able to reflect the value back into the original variable.

For some inspiration check how it works in SeamlessRemoteWorkspaceVariable. In RPlayground remote scripts actually updates the local variables value stored on the client side.

But it can be not very suitable here because we don't have a way to substitute the normal variables (the playground case is special anyway in local playground).