question about brain.py

[rewardisenough]

Hi walker, I don't know if you will see this or not but I have some questions about brain.py.
def dense(X, units, params):
# Reshape the parameters from a flat array to a matrix of weights and biases
reshaped = params.reshape(units, X.shape[1] + 1)
# Apply the weights and biases matrix
return np.dot(X, reshaped[:, :-1].transpose()) + reshaped[:, -1]

For example, if my x data was (700,4) which has 4 features, wouldn't the starting params be (4,) which is a 1d array?
Suppose, I also want 5 output for my neural networks. Then according to your code, reshaped = params.reshape(5,4+1). But this is not possible since you can't reshape an array that's not the same size as the output array(reshaped). 4 is not equal to 25.

Thanks.

[rewardisenough]

Okay, I read the documentation for brain.py and it said,
"A dense linear model. Extra parameter is the number of outputs. Expects a number of parameters equal to the number of outputs, times the number of inputs plus one."

So i guess starting parameters should be something like (25,) so that it can be changed to (5,4+1) when it's being reshaped.

I'm also trying to build a library so I was just curious to know how you did it.

[Hyperdraw]

@rewardisenough

Hi! Sorry I didn't see your question sooner - for some reason it didn't show in my Github notifications until today, and then I had to go study my own code to remember how it works.

Purpose of dense

The dense function is matrix multiplication with the weights and then addition with the biases. Both weights and biases are part of params. units is the number of outputs.

Meaning of params.reshape(units, X.shape[1] + 1)

The goal is to make a matrix with a number of rows equal to units and a number of columns equal to the number of input features, which is X.shape[1]. We also need to include the biases, which for now we'll put as an extra column that we'll break off later. Hence, X.shape[1] + 1 columns puts the biases at the last column.

Meaning of np.dot(X, reshaped[:, :-1].transpose()) + reshaped[:, -1]

np.dot, when applied to two 2D arrays (matrices), does matrix multiplication. We're multiplying X (the matrix of all input data points and all input features) with the weights matrix. The weight matrix is reshaped[:, :-1] because we need to exclude the extra biases column. It is transposed to make the multiplication work correctly. Then we add reshaped[:, -1] to finally add the biases.

What that means for the params

The reason dense "Expects a number of parameters equal to the number of outputs, times the number of inputs plus one" is that for each output, we need a number of weights equal to the number of input features plus one bias. So, yes, for 4 input features and 5 units you will need 25 params.

Conclusion

I hope that helps answer your question. Also, sorry I haven't checked Discord in quite a while. Here's my email address, which I get notifications for: howerd.rex@gmail.com