From deep network mysteries to physics