How React/Redux codebases integrate with Fronts/Clients

[ochameau]

Context: Fission.

For fission we will have to handle multiple targets at once. One per iframe in regular toolbox. One per process in the browser toolbox.
In order to handle this, typically in redux's actions, we will have to call the right target's front method. For now we had only one front, but now we will have many which will be specific to each resource.

I started looking into how we would handle this in all panels and saw that every panel was having a different way to interact with Fronts/Clients.

I think it is a good time to discuss between all panels and try to agree on a unified way to integrate a React/Redux codebase with Fronts and Clients.

Before suggesting anything I would like to first go over the codebase and correctly describe the current architecture of each panel using React and Redux.

I'm especially interested in describing:

• In which layer(s) (React, Redux, Reducers, middleware or other abstraction layer) we are using the Fronts/Client.
• Where do we process Front responses and if we convert/map/translate them.

Debugger

• React components pass around actor IDs of each source. The thread actor ID is stored in the Redux state as PausedState.currentThread.
• Actions are calling "commands" which are the browser specific action implementations (debugger supports both chrome and firefox) .
• Commands receive a thread actor ID coming from the state object and maps it to a thread client via a Map caching clients by actor IDs.
• It looks like there is very little data mapping between what Front returns and what we save in Redux store (There is this function setting default values)

Click to expand

Source object type definition saved in store

type BaseSource = {|
  +id: string, // <== Actor ID
  +url: string,
  +thread: string,
  ...

An action calling a command

export function command(type: Command) {
  return async ({ dispatch, getState, client }: ThunkArgs) => {
    const thread = getCurrentThread(getState()); // <== Get the current thread/target from the state
    return dispatch({
      type: "COMMAND",
      command: type,
      thread,
      [PROMISE]: client[type](thread)  // <== Call the command with the thread actor ID
    });
  };
}

A command calling a thread client method

function resume(thread: string): Promise<*> {
  return new Promise(resolve => {
    lookupThreadClient(thread).resume(resolve); // <== Call the thread client method after looking up for client instance by an actor ID
  });
}
```js
</details>

### New about:debugging
- In most cases React components only manipulate actor IDs (for everything but SW registration front). Components are then calling actions with actor IDs and actions are using multiple ways, each time specific to each actor type, to get the front out of an actor ID.
- Actions are calling front methods and passing the result over to reducers which pass the data as-is, but a middleware is mapping the data to a custom representation.
- about:debugging is the only tool to use middleware to map data to a custom representation.

<details>
  <summary>Click to expand</summary>

[An action call a front method and dispatch a new action with the response](https://searchfox.org/mozilla-central/rev/b29663c6c9c61b0bf29e8add490cbd6bad293a67/devtools/client/aboutdebugging-new/src/actions/debug-targets.js#199-220)
```js
      const {
        otherWorkers,
        serviceWorkers,
        sharedWorkers,
      } = await clientWrapper.listWorkers();
      ...
      dispatch({
        type: REQUEST_WORKERS_SUCCESS,
        otherWorkers,
        serviceWorkers,
        sharedWorkers,
      });

The reducer just pass the data as-is

    case REQUEST_WORKERS_SUCCESS: {
      const { otherWorkers, serviceWorkers, sharedWorkers } = action;
      return Object.assign({}, state, { otherWorkers, serviceWorkers, sharedWorkers });
    }

A middleware is processing the front response

    case REQUEST_WORKERS_SUCCESS: {
      action.otherWorkers = toComponentData(action.otherWorkers);
      action.serviceWorkers = toComponentData(action.serviceWorkers, true);
      action.sharedWorkers = toComponentData(action.sharedWorkers);

And maps it to a custom representation

function toComponentData(workers, isServiceWorker) {
  return workers.map(worker => {
    // Here `worker` is the worker object created by RootFront.listAllWorkers
    const type = DEBUG_TARGETS.WORKER;
    const icon = "chrome://devtools/skin/images/debugging-workers.svg";
    let { fetch } = worker;
    const {
      name,
      registrationFront,
      scope,
      subscription,
      workerTargetFront,
    } = worker;

    // For registering service workers, workerTargetFront will not be available.
    // The only valid identifier we can use at that point is the actorID for the
    // service worker registration.
    const id = workerTargetFront ? workerTargetFront.actorID : registrationFront.actorID;

    let isActive = false;
    let isRunning = false;
    let pushServiceEndpoint = null;
    let status = null;

    if (isServiceWorker) {
      fetch = fetch ? SERVICE_WORKER_FETCH_STATES.LISTENING
                    : SERVICE_WORKER_FETCH_STATES.NOT_LISTENING;
      isActive = worker.active;
      isRunning = !!worker.workerTargetFront;
      status = getServiceWorkerStatus(isActive, isRunning);
      pushServiceEndpoint = subscription ? subscription.endpoint : null;
    }

    return {
      details: {
        fetch,
        isActive,
        isRunning,
        pushServiceEndpoint,
        registrationFront, <==== Front
        scope,
        status,
      },
      icon,
      id,
      name,
      type,
    };
  });

SW registration front is saved in the redux store and used from a React component to hand it over to an action

  dispatch(Actions.startServiceWorker(target.details.registrationFront));

The action calls a front method

function startServiceWorker(registrationFront) {
  return async (_, getState) => {
    try {
      await registrationFront.start();

Otherwise for every other resources we retrieve the front out of the ID, like here for addons:

  const front = devtoolsClient.getActor(id);

Or for SW

  const workerActor = await clientWrapper.getServiceWorkerFront({ id });
  await workerActor.push();

https://searchfox.org/mozilla-central/rev/b29663c6c9c61b0bf29e8add490cbd6bad293a67/devtools/client/aboutdebugging-new/src/modules/client-wrapper.js#119-123

  async getServiceWorkerFront({ id }) {
    const { serviceWorkers } = await this.listWorkers();
    const workerFronts = serviceWorkers.map(sw => sw.workerTargetFront);
    return workerFronts.find(front => front && front.actorID === id);
  }

Or addons

  const addonTargetFront = await clientWrapper.getAddon({ id });
  await addonTargetFront.reload();

Accessibility

• React components are having access to target scoped front (walker/accessibility fronts) via props.
• Actions receive the target scoped front, call a method and reducers translate front response into a custom object.

Click to expand

An action receive the fronts from React props and call a front method. The response is passed to reducers.

exports.updateDetails = (domWalker, accessible, supports) =>
  dispatch => Promise.all([
    domWalker.getNodeFromActor(accessible.actorID, ["rawAccessible", "DOMNode"]),
    supports.relations ? accessible.getRelations() : [],
  ]).then(response => dispatch({ accessible, type: UPDATE_DETAILS, response }))
    .catch(error => dispatch({ accessible, type: UPDATE_DETAILS, error }));

Reducer process the response and maps it to a custom representation

    case UPDATE_DETAILS:
      return onUpdateDetails(state, action);
function onUpdateDetails(state, action) {
  const { accessible, response, error } = action;
  if (error) {
    console.warn("Error fetching DOMNode for accessible", accessible, error);
    return state;
  }

  const [ DOMNode, relationObjects ] = response;
  const relations = {};
  relationObjects.forEach(({ type, targets }) => {
    relations[type] = targets.length === 1 ? targets[0] : targets;
  });
  return { accessible, DOMNode, relations };
}

performance-new

• React components only use front from here so we can almost say that only actions are using front.
• Target scoped front (Perf front) is stored in state object and actions are retrieving it from it.
• There is no particular mapping of data as we don't use much data out coming out from this front.

Click to expand

An action retrieve the target scoped front from the state and call a front method

    const perfFront = selectors.getPerfFront(getState());
    perfFront.startProfiler(recordingSettings);

Details on how we retrieve the front from the state

    const getPerfFront = state => getInitializedValues(state).perfFront;

Application

• React components have fronts set on props and use it directly.
• "initializer", a main startup module/class, populate the store by calling front itself and dispatching action with front responses.
• The actions and reducers pass front data as-is.

Click to expand

React component calling a front method

    const { registrationFront } = this.props.worker;
    registrationFront.start();

initializer module call front method and then an action with the response without any mapping

    const { service } = await this.client.mainRoot.listAllWorkers();
    this.actions.updateWorkers(service);

Memory

• React components store the target scope front on props, call actions with it as argument
• I couldn't find any particular mapping of memory front generated data.

Click to expand

React component calling an action with front coming from props

            dispatch(toggleRecordingAllocationStacks(front)),

Action calling a front method

            await front.startRecordingAllocations(ALLOCATION_RECORDING_OPTIONS);

Netmonitor

• React components as well as actions go through a "Connector" class that itself communicated with the Client/Front
• This Connector listen for client events and do the necessary request to populate state via actions. It does map data to a custom representation and automatically does some request to expand long string for example.

Click to expand

Connector is calling an action with processed/aggregated data comping front Client responses

      await this.actions.addRequest(id, {
        // Convert the received date/time string to a unix timestamp.
        startedMillis: Date.parse(startedDateTime),
        method,
        url,
        isXHR,
        cause,

        // Compatibility code to support Firefox 58 and earlier that always
        // send stack-trace immediately on networkEvent message.
        // FF59+ supports fetching the traces lazily via requestData.
        stacktrace: cause.stacktrace,

        fromCache,
        fromServiceWorker,
        isThirdPartyTrackingResource,
        referrerPolicy,
      }, true);

It does map data coming from RDP like here:

    const payload = {};
    if (responseContent && responseContent.content) {
      const { text } = responseContent.content;
      const response = await this.getLongString(text);
      responseContent.content.text = response;
      payload.responseContent = responseContent;
    }
    return payload;

In is interesting to note a usage of target front usage

    const toolbox = gDevTools.getToolbox(this.props.connector.getTabTarget());
    toolbox.viewSourceInDebugger(url, 0);

An action call the connector

    connector.sendHTTPRequest(data, (response) => {
      return dispatch({
        type: SEND_CUSTOM_REQUEST,
        id: response.eventActor.actor,
      });
    });

The connector is calling the client method

  sendHTTPRequest(data, callback) {
    this.webConsoleClient.sendHTTPRequest(data, callback);

console

• Only one action calls a client method for autocompletion. It fetches the client reference from the "services".
• This "services" is exposed to all actions via a middleware
• And to many React components via "serviceContainer" props.
• JSTerm, WebConsoleFrame and WebConsoleOutputWrapper are all doing request to the client
• One React components craft clients and hand it over to an action to do more requests on the given client.
• Otherwise, the typical flow is to have WebConsoleProxy listen for client events and do the requests and then dispatch actions which are mapping data to a custom representation.

Click to expand

JSTerm is calling a client method

    return this.webConsoleClient.evaluateJSAsync(str, null, {
      frameActor: this.props.serviceContainer.getFrameActor(options.frame),
      ...options,
    });

WebConsoleFrame is calling a client method

  this.webConsoleClient.setPreferences(toSet, response => {
      if (!response.error) {
        this._saveRequestAndResponseBodies = newValue;
        deferred.resolve(response);

WebConsoleOutputWrapper calls a client method

      this.hud.webConsoleClient.clearNetworkRequests();

A React component is instantiating a client class

    const client = new ObjectClient(serviceContainer.hudProxy.client, parameters[0]);
    const dataType = getParametersDataType(parameters);

    // Get all the object properties.
    dispatch(actions.messageTableDataGet(id, client, dataType));

WebConsoleProxy listen to client events and call WebConsoleOutputWrapper

  _onLogMessage: function(type, packet) {
    if (!this.webConsoleFrame || packet.from != this.webConsoleClient.actor) {
      return;
    }
    this.dispatchMessageAdd(packet);

WebConsoleOutputWrapper dispatch actions (with some batching optimizations in the way)

  store.dispatch(actions.messagesAdd(this.queuedMessageAdds));

The action is using a transformation to map the data to a custom representation

There is one action using the console client directly

    client.autocomplete(
      input,
      undefined,
      frameActorId,
      selectedNodeActor,
      authorizedEvaluations
    ).then(data => {
      dispatch(
        autocompleteDataReceive({
          id,
          input,
          force,
          frameActorId,
          data,
          authorizedEvaluations,
        }));

https://searchfox.org/mozilla-central/rev/b29663c6c9c61b0bf29e8add490cbd6bad293a67/devtools/client/webconsole/reducers/autocomplete.js#45-84

[ochameau]

The first iteration of this RFC would be to agree on the description I made of each panel, that, without being argumentative yet.

Could I get a review of at least one peer of each panel?
@nchevobbe for the console? @janodvarko for netmonitor? @jasonLaster for debugger? @julienw for perf/memory? @juliandescottes for about:debugging/application panel?
I you think there is a better way to sort out the actual description, I'm all up for modifying the original message of this PR! May be you are already having some doc describing your panel architecture somewhere? If not, such description could be a nice documentation :)

[MikeRatcliffe]

I know the storage panel will eventually be part of our app panel (or whatever we call it) but I thought I would mention that it handles this in a completely different way.

[nchevobbe]

Thanks for doing this review Alex. That sounds correct for the console and reveals a lot of inconsistencies, that would be nice to unify as part of this work.

[codehag]

thanks for doing this, the findings are really interesting.

[julienw]

About performance-new: remember that the perf front is a root front, because we control the profiler from gecko's main process. Therefore do you think we'll need to have several targets for this one?

About memory: this looks right from what I can see.

[juliandescottes]

Thanks for looking into this Alex!

The description for the application panel looks good.

For about:debugging, it is a bit different from the application panel.

• we store actor ids in the state, not fronts
• we use fronts to call the server in actions (or in shared modules called from actions)
• since we only store ids, we fetch the front dynamically in actions everytime we need to do something on it
• we transform the responses from fronts in middleware files

We are definitely open to change this for more consistency.

I will just share some of the motivations behind the current choices (and maybe others can point at better patterns to address them). One of the strong drives was to have similar looking data structures for all our debug targets so that we could build generic components to display them. Initially not all of our debug targets had fronts as well so using the actor id was the only consistent way to store an access point to the server.

Finally we have a "client-wrapper" class to avoid calling the client directly from our application code, so that we can have a layer which is easy to mock. Open to any other solution (service container? commands?) that still preserves the ability to mock this easily in browser mochitests.

(You asked whether we had documents about architecture. The only thing I have are:

• https://docs.google.com/document/d/11CRu0F6GvviATF1FK5IwrJH0KtepLtJdNNFWPhYg7lo/edit#heading=h.aisn0hafb0jg discussion document about react panel architecture. We had a meeting with Honza and Nicolas before starting the application panel to try and identify patterns we should reuse in the application panel for consistency. We reapplied the same for about:debugging.
• https://searchfox.org/mozilla-central/source/devtools/client/aboutdebugging-new/README.md some very high level architecture documentation for about:debugging)

[jasonLaster]

Thanks Alex!

I think the debugger does two smart things:

1. Stores resource actor IDs in redux (sources, breakpoints, frames, …)
2. Use a client API in redux actions to talk with the server via fronts/clients

The redux actions receive the client via makeThunkArgs which is a style of dependency injection.

configureStore({
    log: prefs.logging,
    timing: isDevelopment(),
    makeThunkArgs: (args, state) => ({ ...args, client })
  });

Separating the client API has a couple of advantages:

1. Our redux unit tests stub the client, dispatch actions, and assert selector responses. These tests are really expressive. perf.html does something similar.

  it("should add an expression", async () => {
    const mockClient = { evaluateInFrame: async expression => `blah` };
    const { dispatch, getState } = createStore(mockClient);
    await dispatch(actions.addExpression("foo"));
    expect(selectors.getExpressions(getState())).toEqual([{ input: "foo", value: "blah" ]);
  });

2. It is easy to refactor/debug the client/server communication because it is in one place.

stepOver is a fairly simple client method.

function stepOver(): Promise<*> {
  return threadClient.stepOver();
}

When Brian added multi-target debugging we were able to update stepOver to first lookup the thread client and then call stepOver.

async function stepOver(thread: string): Promise<*> {
  return lookupThreadClient(thread).stepOver();
}

Keeping the RDP logic like threadClient in the client module was the primary reason we were able to make the debugger multi-target in a couple of weeks.

---

addendum: chrome cdp

The debugger currently ~50% cdp, which makes it easy to compare CDP and RDP.

Here is an example of chrome's setBreakpoint, vs Firefox's setBreakpoint.

async function setBreakpoint(location: SourceLocation, condition: string) {
  const { breakpointId, serverLocation } = await debuggerAgent.setBreakpoint({
    location: toServerLocation(location),
    columnNumber: location.column
  });

  return {
    id: breakpointId,
    actualLocation:  fromServerLocation(serverLocation) || location
  };
}

function setBreakpoint(
  location: SourceLocation,
  options: BreakpointOptions,
): Promise<BreakpointResult> {
  const sourceThreadClient = sourceThreads[location.sourceId];
  const sourceClient = sourceThreadClient.source({ actor: location.sourceId });

  return sourceClient
    .setBreakpoint(location, options)
    .then(([{ actualLocation }, bpClient]) => {
      actualLocation = createBreakpointLocation(location, actualLocation);
      const id = makePendingLocationId(actualLocation);
      bpClients[id] = bpClient;
      return { id, actualLocation };
    });
}

The big difference is that Chrome's API accepts IDs and does not manage resource instances.

[yzen]

Thanks Alex, looks pretty accurate for the Accessibility panel.