Dissecting Deep Learning Networks-Visualizing Mutual Information.

In the LeNet, an increasing trend of MIs during the training process is not obvious, except in the very early stages of training, which is contrasted to the first block of the model (see Figure 5 a).

also in 63,971 other papers

Over training, some kernels start producing high MI values, but there is a decreasing trend of MI values after 20,000 iterations, consistent with Figure 8 b.

also in 47,180 other papers