Specify the cutlass codegen for TVM

[YukeWang96]

How does this part of code specify the cutlasscodegen for code generation?

tvm-cutlass-eval/resnet50/run.py

Lines 30 to 39 in 6684423

	mod = partition_for_cutlass(mod, params)
	print(mod)
	mod, num_cutlass_partition = tune_cutlass_kernels(
	mod, sm, profile_all=True, use_multiprocessing=True, tmp_dir=tmp_dir
	)
	with tvm.transform.PassContext(opt_level=3):
	lib = relay.build(mod, target="cuda", params=params)
	lib = build_cutlass_kernels(lib, sm, tmp_dir, lib_path)
	rt_mod = tvm.contrib.graph_executor.GraphModule(lib["default"](dev))
	return rt_mod, dev, num_cutlass_partition

[masahi]

It is done by partition_for_cutlass.
https://github.com/apache/tvm/blob/e7f36487dfdb6c4b7b544be155d3869002d7281b/python/tvm/relay/op/contrib/cutlass.py#L175

Each pattern is prefixed with cutlass. and transform.AnnotateTarget(["cutlass"]) tells the graph partitioner what ops to partition.

[YukeWang96]

Thanks for your quick reply, I encounter another problem, after using annotateTarget on the defined mod

AssertionError: annotation.compiler_begin doesn't have an FTVMStrategy registered. You can register one in python with `tvm.relay.op.register_strategy`.

is this because of a missing strategy for the operator?

[masahi]

I don't think so. annotation.compiler_begin is not a real op, so there is no strategy for it. This op is supposed to disappear after you run PartitionGraph, have you run it?

[YukeWang96]

May I know which line or code snippet eliminates this annotation.compiler_begin.
Does this line make difference?
https://github.com/apache/tvm/blob/e7f36487dfdb6c4b7b544be155d3869002d7281b/python/tvm/relay/op/contrib/cutlass.py#L256

[masahi]

As I said above, it's PartitionGraph's job https://github.com/apache/tvm/blob/e7f36487dfdb6c4b7b544be155d3869002d7281b/python/tvm/relay/op/contrib/cutlass.py#L258

[YukeWang96]

It works, thanks a lot!