Evaluation passed and the next step: OpFromGraph

The PR of function.copy() is ready to merged, only need fred to fix a small bug. And in this Friday I passed the mid-term evaluation. So it’s time to take the next step.

In the original proposal ,the next step is to swap output and updates. After a discussion with Fred, we thought this feature is useless so we skip this and head to the next feature directly – OpFromGraph.

Goal:

make class OpFromGraph work.

Big How?

OpFromGraph should init a gof.op that has no difference with other Ops and can be optimized. Otherwise it has no sense.

For this, we need to make it work on GPU, make sure it works with C code and document it. Make sure infer_shape(), grad() work with it. Ideally, make R_op() work too.

Detailed how.

• Implement __hash__() and __eq__() method so it is a basic
• Implement infer_shape() method so that it’s optimizable
• test if it work with shared variable as input and if not make it work. Add test for that.
• Move it correctly to the GPU. We can do it quickly for the old back-end, move all float32 inputs to the GPU. Otherwise, we need to compile the inner function, see which inputs get moved to the GPU, then create a new OpFromGraph with the corresponding input to the GPU. #2982
• Makergrad() work. This should remove the grad_depth parameter

First Step: infer_shape:

The main idea is to calculatet the shapes of outputs from given input shapes. This is a process similar to executing a function – we cannot know the shape of a variable before knowing the shape of the variables it depends on. So, we can mimic the make_thunk() method to calculate the shape from output to input. I come out with a draft now, and need some help with test case.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

order = self.fn.maker.fgraph.toposort()

# A dict that map variable to its shape(list)
shape_map = {}

# set the input shape of the fgraph
for in_var, shape in izip(in_vars, shapes);
    shape_map.set_default(in_var, shape)

# calculate the output shape from input shape
for node in order:
    assert all([var in shape_map.keys() for var in node.inputs])

    # calculate output shape
    in_shapes = [ shape_map[var] for var in node.inputs]
    out_shapes = node.op.infer_shape(node, in_shapes)

    # store the shape of that variable
    for out_var, shape in izip(node.outputs, out_shapes):
        shape_map.set_default(out_var, list(shape))

# extract output shape
return [ tuple(shape_map[var]) for var in fgraph.outputs]