A network 20 layers deep starts with random weights. Each unit adds up inputs_per_unit inputs, each times its own random weight, and with relu=True a ReLU (“Stacking straight lines”) follows every layer. Write weight_size(inputs_per_unit, relu): the typical size of a starting weight so a signal comes out of 20 layers about as big as it went in, between half and double.