Deep learning is widely used for seismic impedance inversion, but few work provides in-depth research and analysis on designing the architectures of deep neural networks and choosing the network hyperparameters. This paper is dedicated to comprehensively studying on the significant aspects of deep neural networks that affect the inversion results. We experimentally reveal how network hyperparameters and architectures affect the inversion performance, and develop a series of methods which are proven to be effective in reconstructing high-frequency information in the estimated impedance model. Experiments demonstrate that the proposed multi-scale architecture is helpful to reconstruct more high-frequency details than a conventional network. Besides, the reconstruction of high-frequency information can be further promoted by introducing a perceptual loss and a generative adversarial network from the computer vision perspective. More importantly, the experimental results provide valuable references for designing proper network architectures in the seismic inversion problem.