�ݺ�ߣ

Editor's Notes

• #4: Title: Toward new definitions of equivalence in verifying deep learning compilers Author name: Takeo Imai Author affiliation: LeapMind Inc. (Currently, he is a freelance engineer and also at National Institute of Informatics) Abstract (word count: 373/500): A deep learning compiler is a compiler that takes a deep neural network (or DNN) as an input, optimizes it for efficient computation, and outputs code that runs on hardware or a platform. Optimizations applied during the compilation include graph optimizations like operator fusion and tensor optimizations like loop optimizations for matrix multiplication and accumulation. In addition to those classical optimizations, a deep learning compiler often applies optimizations specific to deep learning accelerators. One common example is quantization, which reduces the bit length of parameters and its computations in a DNN. A compiler may quantize 32bit float values into n-bit integer, where n = 8 is most common and n = 1 or 2 for some specific hardware devices. In this talk, we shed light to the difficulties in defining what is equivalence for deep learning compilers. For compilers of ordinary programming languages, the behavior of a program before/after compilation must be equivalent, regardless of optimization passes applied in the compilation process. It is commonly understood that having the ��same�� behavior is not to have exactly the same output values from the same input, and an output value including some tiny errors like rounding errors in floating point operations are generally accepted, according to the ordinary equivalence criteria. A deep learning compiler, however, sometimes produces a code that does not keep the equivalence in a classical sense; for example, a tiny rounding error caused at a hidden layer may change the value from 0 to 1 after a 1-bit quantization, which may bring a completely different final classification result compared with the original DNN��s behavior. This is because the final output of a DNN is discrete-valued .The classical equivalence criteria do not take into consideration such tiny-error, big-difference cases. This is a fundamental issue in the equivalence verification of deep learning compilers. We consider that we need to start from redefining the equivalence of DNN computation, or ��relaxing�� the equivalence criteria in order that some difference of individual discrete-valued results can be acceptable. And then, we need to propose new testing or verification methods according to the new equivalence criteria. We present issues around the correctness of deep learning compilers described above, and offer a direction for our future work about DNN compilation.
• #20: include/tvm/relay/type.h src/tvm/relay/op/nn/convolution.cc