Frame 105:25 (plane function)

[Solaxun]

Hi all! Going through the book and got a bit stuck on this frame. The fn plane says it takes a tensor1 arg t, but in frame 110:38 it's called (through l2-loss) with the dataset in frame 104:23 where xs is a tensor2.

Is this a typo or am I misunderstanding?

Separate question:
Frame 50:14 shows the example:
(+ [6 9 1] [[4 3 8] [7 4 7]])

which results in:

[[10 12 9] [13 13 8]]

This is in-line with running the following code:

(+ (tensor 6 9 1)
   (tensor
    (tensor 4 3 8)
    (tensor 7 4 7)))

However, if you extend (heh) this example by adding another t1 to the second tensor, things change.
(+ [6 9 1] [[4 3 8] [7 4 7] [5 9 8])

Here I would expect a result, similar to the previous example, of:
[[10 12 9] [13 13 8] [11 18 9]

but instead we get:
[[10 9 14] [16 13 16] [6 10 9]]

which can be seen using the following code:

(+ (tensor 6 9 1)
   (tensor
    (tensor 4 3 8)
    (tensor 7 4 7)
    (tensor 5 9 8)))

So instead of the first argument broadcasting to each t1 in the second argument, each scalar in the first argument is broadcasting to each t1 in the second argument. Is this expected?

[themetaschemer]

Hi @Solaxun. Great questions, both.
For the first, since plane is defined with extended operators, it can operate on tensor2's. It will produce a tensor1 as a result (instead of a scalar). We make use of this property throughout the book.

The second question is also a very good one and has a somewhat subtle answer.

In general, when you try to add two tensors of the same tlen, for example:

(+ (tensor A0 B0 C0) (tensor A1 B1 C1))

We expect the result to be:

(tensor (+ A0 A1) (+ B0 B1) (+ C0 C1))

Now, if A0, B0, and C0 are tensor0 (i.e., scalars) and A1, B1, and C1 are tensors1, (as they are in your example), you will get the result that you got, which is [[10 9 14] [16 13 16] [6 10 9]]

The alternative result that you expected would have come from:

(tensor (+ (tensor A0 B0 C0) A1) (+ (tensor A0 B0 C0) B1) (+ (tensor A0 B0 C0) C1))

which is somewhat counterintuitive. There is a way to get this behavior as well by defining:

(define +-1-1 (ext2 + 1 1))

This will then get you your desired behavior:

> (+-1-1 (tensor 6 9 1) (tensor (tensor 4 3 8) (tensor 7 4 7) (tensor 5 9 8)))
(tensor (tensor 10 12 9) (tensor 13 13 8) (tensor 11 18 9))

(note: Due to a bug in the flat-tensors representation, which I just found out, this last extension currently only works for learner and nested-tensors representations. I am working on a fix for flat-tensors, and will post an update here shortly.)

[Solaxun]

Thanks for the reply!

Yeah I did notice that plane works with tensor2 via extended functions, I was just confused by the footnote in 105:25 which says that the argument t is a tensor1 since at 110:38 it's called with a tensor2. I think it would be more clear to omit the tensor1 and just say that plane takes a tensor.

For the second question I think I get your point, but it is definitely counter-intuitive (for me at least).

So in this example:
(+ (tensor A0 B0 C0) (tensor A1 B1 C1))
if A0, B0, and C0 are scalars, then you get:
(tensor (+ A0 A1) (+ B0 B1) (+ C0 C1))
because the tlen is the same

However in this example (assuming again A0, B0, and C0 are scalars):
(+ (tensor A0 B0 C0) (tensor A1 B1))
You get:
(tensor (+ (tensor A0 B0 C0) A1) (+ (tensor A0 B0 C0) B1))
because the tlen is not the same.

That is different than the way I've seen broadcasting work elsewhere, but maybe it's just an implementation specific thing - e.g. Python's Numpy:

x = np.array([6,9,1])
y = np.array([[4,3,8],[7,4,7],[5,9,8]])
x+y
>>> array([[10, 12,  9], [13, 13,  8], [11, 18,  9]])

[themetaschemer]

Yes, that's a good point. We will clarify it further in the book.

You are also right that the choice of how to do the extension/broadcast is somewhat arbitrary and a case can be made to go in either direction.

[themetaschemer]

The bug discovered for the flat-tensors implementation has now been fixed in c5f05d8.