softmax 求导 / label_smoothing 求导 / 知识蒸馏求导

softmax求导y^i=softmax(zi)=exp⁡zi∑j=1Kexp⁡zjloss=−∑i=1Nyilog⁡y^i\hat{y}_i=\text{softmax}{(z_i)}=\frac{\exp{z_i}}{\sum_{j=1}^K\exp{z_j}}\\loss=-\sum_{i=1}^Ny_i\log\hat{y}_iy^​i​=softmax(zi​)=∑j=1K​exp...